Self-Supervised Learning for Wireless Localization

1.1 Wireless localization with deep learning

Wireless localization methods can be divided into two categories: model-based and data-driven [3]. Model-based techniques require knowledge of the geometric relationship between the estimated parameters of the received signal and the position of the transmitter [4]. Therefore, the performance of the existing model-based methods is heavily degraded when such a relationship is not available, for example, in non-line-of-sight (NLOS) or dense multi-path propagation conditions [4, 5, 6, 7]. Consequently, data-driven approaches, specifically ML, have emerged over the years [8]. Among the various ML approaches, deep neural network (DNN)-based models have demonstrated exceptional localization accuracy [9, 10, 11, 12, 13, 14, 15].

A common approach in cellular-based localization systems is to use derived features of the estimated channel at the base station (BS), such as signal time of arrival (ToA), angle of arrival (AoA), received signal strength (RSS), or a combination of them [6, 16, 17]. On the other hand, DNN-based methods prefer utilizing the channel state information (CSI) in its acquired form [9, 10, 13, 18, 19]. The acquisition of CSI is particularly important for advanced communication systems that use massive multiple-input multiple-output (MIMO) [20, 21]. Hence, CSI is, in general, readily available for localization. Massive MIMO, characterized by a large antenna array at the BS, is widely regarded as the key technology for the fifth-generation (5G) [22, 23] and forthcoming communication systems [24]. Employing a considerable number of antennas enhances the angular resolution of the received multipath signal, thereby benefiting localization methods [25, 26, 27].

ML-based methods can generally be supervised or unsupervised, depending on whether labeled training data are necessary. A vast majority of DNN localization techniques are supervised and constrained to a task-specific feature learning, raising concerns particularly about the ability to work across diverse scenarios and adapt in data-scarce environments. On the other hand, developing DNN methods that sustain good accuracy and transferability is an essential research topic for future wireless localization systems.

In this chapter, we cover three approaches for wireless localization. We investigate conventional dimensionality reduction (DR) techniques, like PCA and basic manifold approaches. Afterward, we discuss a supervised learning method that uses multi-layer perceptrons (MLP). Furthermore, we extend the basic MLP into a deep metric learning method. Finally, we transition into advanced DNN architectures such as transformers, combined with sophisticated learning frameworks like self-supervised learning, emphasizing its growing research importance in wireless localization and beyond.

Specifically, we structure the rest of this chapter as follows. First, in Section 2, we outline the system model to generate synthetic datasets for validating DNNs. In Section 3, we investigate classical DR approaches. Then, in Section 4, we discuss supervised DNN methods and deep metric learning. In Section 5, we elaborate on more advanced neural network architectures and learning frameworks. Finally, we draw our conclusions in Section 6.

2.1 Signal model

The uplink input-output relationship of a single-user transmission from ur on subcarrier n∈1…Nc is

Here, Ptx is the average transmit power, xn is the normalized transmitted signal with xn2=1, h˜n∈CNr×1 is the channel vector, and nn∼CN0N0WscINr, where N0 is the noise power spectral density and Wsc is the subcarrier bandwidth. We assume that the BS is able to estimate the channel vectors h˜n through uplink pilot signals. We denote the estimated CSI as hn.

2.2 Channel model

The DNN techniques detailed in this chapter are data-driven and therefore are not constrained to a particular channel model. Nevertheless, it is beneficial to clarify the channel using location-related parameters like distances and angles for investigation. Consequently, we utilize a commonly adopted geometric channel model to characterize the estimated channel of the received signal for such scenarios as that illustrated in Figure 1 [28],

where ηg and τg represent the complex gain and propagation delay (ToA) of the g−th path between the r−th user equipment (UE) position and the m−th BS. The angles of arrival (AoA) for azimuth and elevation are represented as φaz,g and φel,g, respectively, while the subcarrier spacing is denoted by Δf. Assuming a uniform planar array with antennas arranged along the x− and z−axis, the array response vector at the receiver is expressed as aφazφel=azφel⊗axφazφel, where ax⋅ and az⋅ are given as

Here, λc=cl/fc, where fc is the carrier frequency, and cl denotes the speed of light. The antenna element spacing is given by d=λc/2.

2.3 Dynamic scenario

We consider that the environment may change over the time. When using synthetic data to evaluate the proposed methods in this chapter, we account for situations where some scattering objects change their positions over the time interval T, assuming, for example, a dynamic setting [29]. We formulate pg,it=pg,i+wg,i, with wg,i representing the Gaussian noise with zero-mean and variance σw2 at the i−th coordinate. Doing so leads to a non-additive distortion model [24], where the angle, delay, and channel gain of the individual multi-bounce NLOS paths change for each channel realization. We also assume the uncertainty in the position of antenna for the UE, modeled as ur,it≜ur,i+ẇr,it. Similar to the uncertainty in the position of the scattering objects, ẇg,i is the Gaussian noise with variance σẇ2 at i−th coordinate. The variability in antenna position serves as a means to address potential imperfect channel estimates and other system impairments, which may arise from, for instance, a lack of perfect synchronization between the UE and the BS.

Next, we stack Nc′≤Nc subcarriers in a matrix form for the r−th UE location,

where Nc′ may correspond to pilot subcarriers. The input to a neural network-based model is real-valued; hence, we handle complex-valued channel coefficients by stacking their real and imaginary parts and, in some of our architectures, their absolute values. Throughout the chapter, we omit the index t from the expressions as we consider a single-time snapshot in the algorithms presented in here. Similarly, we drop r if not relevant to the discussion or analysis. Finally, the localization performance in this chapter is reported in terms of the root-mean squared error, RMSE,

where Rtest denotes the number of test locations, and ûr′ is the estimated UE location.

3.1 Principle component analysis and iterative scaling

Since principle component analysis (PCA) is the most widely used technique for lossy data compression or feature extraction, we first illustrate the efficacy of the PCA-derived channel subspace when used to determine the UE location. To do so, let us consider a single-path line-of-sight (LOS) channel, that is, G=1 in (2). We also consider a single subcarrier n=1 and assume a uniform linear array (ULA) at the BS. Therefore, we assume that the channel coefficients of hr are dependent only on the distance ∥bm−ur∥ and the direction of arrival φaz. Furthermore, we consider that a dataset from R distinct UE locations is available, construct an R×Nr matrix,

and center it, that is, H¯ref=Href−11RHrefT1T is the zero-mean sample channel matrix, and 1 is a column vector of ones.

Our goal is to construct a representation map of (pseudo-)locations, Z=zr∈RDr=1R, only from the high-dimensional CSI samples. The derived map can be thought of as extracting a low-dimensional representation of UEs spatial information from corresponding high-dimensional channel vectors hrr=1R. We can compute the PCA to get a low-rank approximation of the channel from R locations by eigenvalue decomposition on the empirical covariance matrix RHref=UΣUT, where U are the left singular vectors of H¯ref and the eigenvectors of RHref. Similarly, Σ are the eigenvalues of RHref, that is, squared singular values of H¯ref. The low-dimensional map is considered as

matching the first D largest eigenvalues from σ1…σR with the corresponding eigenvectors urr=1R.

As an optimization problem, PCA is closely related to the so-called multidimensional scaling (MDS) [30], or metric MDS. MDS is another reconstruction method to obtain a representation map based on the dissimilarities between the points. This is particularly appropriate when such dissimilarities can, exactly or approximately, be expressed in terms of the Euclidean distances. Classical scaling, that is, metric MDS, like PCA, results in maximizing the variance in a reduced-dimensional space, which is a widely recognized relationship between the two [31, 32].

In addition to using PCA to construct a low-dimensional map representation by optimizing a convex objective using eigendecomposition, we next look into an alternative to classical scaling, which is known as MDS with Sammon mapping [33]. Sammon mapping is an iterative, gradient-based approach to evaluate a non-convex objective function and is considered a generalization of metric MDS. In this case, we seek to find an optimal representation by normalizing the squared errors using the pairwise distance in the original features space. To obtain the channel features while reducing the distance between low- and high-dimensional representations, we minimize the cost function.

Assuming a subset of CSI has labels, its low-dimensional features can serve as either a reference map or an input to a task-specific model to derive the final location of the UEs. Below, we aim to capture the CSI manifold and investigate the localization performance when CSI is projected onto, for instance, a D=2 dimensional channel map. Provided such maps only yield pseudo-locations, we use a matching algorithm like k-nearest neighbors (k-NN) to compare a new transmitter’s channel features to those of known locations in the reference map. Specifically, we form a distance metric between the features of the unknown location zr′ and all reference locations zr for each r as

and choose rNN=argminrdzr. Finally, the known location with the minimum distance is selected as the estimated position of the transmitter, ûr′=urNN.

Acquiring the reference map poses a challenge in determining the optimal number of low-dimensional features, D. In terms of minimum error formulation, using D=rankHref would, naturally, result in a reconstruction error of zero. However, this does not necessarily ensure a minimized localization error. Furthermore, our objective is to attain D≪Nr. A straightforward approach to select D is to quantify the fraction of variance explained by the selected eigenvalues in H¯ref, that is, ∑i=1Dσi/∑i′=1Rσi′, for example, choosing D to capture 90% of the energy. Alternatively, we can directly evaluate the localization error while sweeping over the value of D. To illustrate the capability of classical DR techniques in retaining low-dimensional features useful for the localization task, in the following, we describe a numerical experiment to evaluate the impact of D.

3.2 Location estimation using D-dimensional features

We set up a simple two-dimensional ROI with a layout of 40m×40m and assume that the user locations are distributed based on a Poisson point process (PPP) with the density λr. Hence, the probability of having R users in the plane is PRA=λrSAre−λrSA/R! [34], where SA is the area of the bounded ROI, A. We place the BS at the origin 00 with respect to y−axis. The BS employs a large ULA with Nr=64. The carrier frequency considered is 3.5 GHz; the pathloss exponent is ρ=2, and the SNR is set to 30 dB to account for a channel estimation error. We assume that there are λr=0.05,0.15,0.30,0.50 users/m2 and keep 100 user locations for testing data. In Figure 2a, we observe that the performance of PCA improves significantly when increasing D from 2 to 4, but then for larger D, it saturates. On the other hand, the iterative MDS with Sammon mapping approach is more robust against changes in the dimensionality, maintaining a similar performance irrespective of the value of D. Yet, the accuracy hardly improves for D>4. We also investigate the localization accuracy while varying the density of reference points (RPs) and keeping D=4. As one can expect, and as shown in Figure 2b, increasing the density of known locations improves the localization performance for both PCA and MDS.

In the next section, we discuss wireless localization with neural networks in general and deep metric learning in particular.

4.1 Metric learning DNN and contrastive task

In contrast to the DR approaches we discussed in Section 3, in deep metric learning, we obtain an embedding space using neural networks. Consequently, the objective of the DNN becomes to learn a D-dimensional embedding that discriminates dissimilar point clouds while simultaneously bringing similar ones closer together. The DNN can map the estimated channel into a metric-defined subspace. Within this space, any distance metric d..:RD→R can be applied to match the derived embedding with the closest RP. Furthermore, we can also incorporate a distance metric-based loss function directly in the embedding space. Among the most recognized deep metric learning methods are those based on the Siamese networks [38]. Such methods have been proposed as feature extractors across different domains, like the works extracting features from images of faces [39]. DNN methods, founded on Siamese architecture, consist of multiple equivalent networks sharing identical weights. Similarly, we introduce a Siamese-based network for obtaining low-dimensional channel representations useful for localization [40], depicted in Figure 4.

The number of networks used in a Siamese-based method can be any. However, it is usually two or three. A Siamese architecture with three equivalent networks is also called a triplet. Hence, the name triplet network for the model. Specifically, the network is composed of three branches. Each network employs the same hyperparameters as the base DNN for the intermediate computation layers. For the final layer (i.e., the output), the identity function is utilized, σzi=zi. The goal of the triplet network is to learn an embedding in the way that the parameters of each network result in the decrease of variance between the channel obtained within a sub-region and increase otherwise. In other words, our goal is to find an objective function that promotes similarity between the embeddings of two CSI samples acquired from the same region r∈1…Rrps while pushing apart embeddings corresponding to CSI samples from different regions.

4.1.2 Impact of LOS and density of reference locations

In the simulations conducted, we choose the model that demonstrates the lowest validation loss over 100 epochs. If the LOS is not present, the margin of the triplet loss during the training process is adjusted to δtri=10. We consider 80% of the dataset for training and the remaining for validation and testing. The scenario covers a 40m×40m area with the BS positioned at the origin 00, relative to the y− axis. Users are spread in the far field, at least 20m away from the BS. The NLOS paths are set to G=3, and the total subcarriers are Nc'=512.

The desired positioning accuracy is primarily determined by the density of RPs. Increasing RPs, the system’s ability for higher accuracy increases, too. However, the presence or the absence of a LOS path in a multipath channel also influences the accuracy of the prediction. Consequently, it limits the accuracy even when a high density of RPs is used.

In Figure 5, we show the impact of LOS with the increased density of RPs for both DNNs, the classifier and the triplet network. In the presence of a LOS path, a higher density of RPs enhances the overall location estimation accuracy. The triplet network surpasses the base DNN classifier when λr>0.05. Conversely, in the absence of the LOS path, a denser RPs distribution results in a degradation of the localization accuracy compared to scenarios where the LOS path is available.

5.1 Wireless transformer

As presented in [29], and depicted in Figure 6, WiT is a fully supervised technique and is trained to minimize

In contrast to prior DNN methods, we view the input channel Hr∈CNr×Nc′ as a series of subcarriers. Each of these subcarriers, denoted by hn∈R1×3Nrn=1Nc′ includes the real, imaginary, and absolute elements of the row channel vectors, that is, 3Nr. Further, individual subcarrier representations undergo a linear embedding process. For each subcarrier, we apply a linear transformation to convert it into an embedding using a linear layer. This layer has learnable parameters, E∈R3Nr×D, leading to the embedding representation ei=hiE.

Transformer-based models, due to their lack of recurrence or standard convolutional operations, treat all subcarrier representations as permutation invariant, ignoring the sequence order of frequency-dependent subcarriers. To address this limitation, we incorporate positional encodings to represent the sequence position of each subcarrier. More specifically, we add a random and learnable real-valued vector embedding, gi∈R1×D, to each subcarrier index i. Subsequently, given the representation of the channel, we combine gi with the i−th embedding. The resulting subcarrier representation is êi=ei+gi, which serves as the input to the subsequent transformer block.

As for the input to the MLP head, which follows the transformer block, we either average the derived features or utilize an extra symbol, LID. This symbol is analogous to the CLS token in [49]. The designated symbol LID represents a trainable vector, that is, ê0∈RD. WiT uses ê0 to learn a compressed representation over all subcarriers. Moreover, the embedding o¯0∈RD is input into a fully connected linear layer, as illustrated in Figure 7b, to map the channel features to location coordinates. The size of the set of input vectors to the transformer block is C=Nc′+1.

5.2 Self-supervised wireless transformer

In contrast to other studies in wireless localization, in this part of the chapter, we discuss our approach to exploiting redundant and complementary information across the subcarriers. Doing so enables us to predict the channel from a single realization. While it might seem counterintuitive to learn to predict the information we possess, we later demonstrate that such an approach can be suitable for deriving meaningful representations for estimating different wireless communication tasks. In contrast to the triplet network detailed in Section 4.1, where we aim to distinguish channels from different sub-regions, here we avoid the necessity of sampling negative pairs and employing a contrastive loss. Furthermore, we show how to leverage the microscopic fading characteristics by designing subcarrier-level pretext tasks.

In the following, we use SWiT to derive a channel representation, denoted as o¯r, without relying on labels or a contrastive objective function. The components of SWiT [51], depicted in Figure 8, are organized in a dual-branch neural network architecture. The first module, that is, channel transformations, comprises a set of stochastic augmentations represented as Q≔Q1…QA, each accompanied by its occurrence probability. Furthermore, we construct TiHr=QA∘QA−1∘…Q1Hr. The goal of the stochastic augmentation module is to produce multiple views of the channel, each accounting for different parts of it. Specifically, the first view is h¯n′∈R3Nrn=1Nc1≜T1Hr, and similarly, the second view is where Nc1 denotes the total number of selected subcarriers for the first transformation. The channel transformation module generates additional Ns random views with fewer subcarriers than the first two. As a result, we obtain V=2+Ns views of an input channel realization. The first two views are referred to as global views, and the others as local.

Next, all channel views are processed sequentially via the encoders fΘ⋅ and fΨ⋅. We refer to the encoders as online and target encoder, respectively. This yields the respective representations for the first view, o¯n∈RDn=1C≔fΨh¯n′n=1Nc1, and similarly for the others.

Given that the wireless channel exhibits both macroscopic and microscopic fading, our method is tailored to address this behavior. Therefore, the learning is split into two separate projectors, namely, micro-fading and macro-fading channel representation learning modules.

5.3 Datasets and self-supervised training

We use real-world channel measurements and synthetic data to further benchmark the performance of the SWiT.

For actual measurements, we selected the ultra dense indoor MaMIMO [13]. This collection has multiple datasets obtained in different massive MIMO setups and propagation conditions in a laboratory. Three specific datasets from this collection were utilized: one featuring NLOS propagation with a uniform rectangular array setup (KUL-NLOS-URA-Lab), another in a LOS and a uniform linear array (KUL-LOS-ULA-Lab), and the third in a LOS and a distributed antenna system setup (KUL-LOS-DIS-Lab). Common parameters across these datasets include Nr=64, Nc′=100, fc=2.61GHz, and R=250,000.

We also choose a synthetic dataset generated based on the discussion in Section 2.2. The datasets are referred to as S-200 and HB-200 [29], representing two dynamic railway scenarios, each with T=200. The S-scenario is characterized by M=1 and R=69212. Conversely, the HB-scenario is characterized by M=8 RRH and a larger sample size of R=81200. For both scenarios, the common parameters are Nr=64, fc=3.5GHz, G=4, and Nc′=32.

5.4 Localization performance and transferability

To assess the performance of learned representations, we perform linear and fine-tuning evaluation for the trained models on several localization tasks. We also investigate the transferability of the models to other tasks and datasets.

5.4.1 Localization accuracy with limited data

For linear analysis, we train a regressor gΦ:RD→RDout atop frozen features from fΨ⋆⋅. For localization, Dout=2. We utilize a single-layer MLP and train for 500−1000 epochs with B=128 and SGD at ϖ=0.03. Additionally, we evaluate the quality of the embedding with a k-NN classifier on frozen fΨ⋆⋅. Performance metrics include top-1 and, when suitable, top-5 accuracy.

To assess the performance of the embeddings, we also perform fine-tuning using labeled data. Specifically, the backbone (i.e., target encoder) is initialized with the pre-trained weights, forming the new network gΦ∘fΨ⋆H. Parameters are set as B=512 and ϖ=3e−4, using AdamW with standard settings. We report accuracy on 32 subcarriers after normalization. A codebase, together with the evaluation results, are publicly available¹.

In Figure 9, we highlight the improvements of SWiT in a small data regime for the case of KUL-NLOS dataset. In contrast to the analysis in [51], we offer additional insights here, highlighting when a simple linear approach is sufficient and when the computationally demanding WiT and SWiT are justified. For the data regimes in the figure, we investigate the RMSE separately for six different randomly sampled datasets. Datasets for evaluation are sampled based on the Poisson point process (PPP) with varying density parameter values λr=0.1,0.5. A value of λdata=1.0≈4500 training examples. For testing, we sample Rtest=5000. We observe that when R≈400 samples, even the linear model, training on top of the channel features learned from SWiT, can outperform a fully supervised model, like WiT, which in here corresponds to the encoder of SWiT.

5.4.2 Spot-localization and transfer learning

Table 2 reports the model’s ability to cluster channel representations while maintaining spatial neighborhood. The KUL datasets, divided into four sub-regions, have Ĉspot=4 spots, while S-200 and HB-200 possess Ĉspot=360 and Ĉspot=406, respectively. The SWiT encoder achieves nearly 100% classification accuracy for KUL datasets, indicating its superior representation learning capability compared to WiT with random weights. In the same table, we also show the transferability (SWiT + TF) of a model to other datasets. More specifically, the trained model on KUL-NLOS dataset is evaluated across other datasets, emphasizing its ability to adapt to diverse MIMO setups and propagation conditions.

	KUL-NLOS		KUL-LOS		KUL-LOS-DIS	S-200		HB-200
Method	↑ Top-1		↑ Top-1		↑ Top-1	↑ Top-1	↑ Top-5	↑ Top-1	↑ Top-5
Random	23.7		3.59		4.13	0.296	1.545	0.27	1.25
SWiT	99.99		99.99		99.99	18.70	52.38	37.38	82.85
SWiT + TF	99.99		99.97		99.98	16.1	44.34	20.68	53.58

Table 2.

Random weights versus SWiT.

In Figure 10, we show the two-dimensional t-SNE [53] embeddings of a merged KUL dataset, encompassing datasets in LOS and NLOS conditions, resulting in a total of Ĉspot=8 spots. For LOS conditions, both randomly initialized WiT and pre-trained models exhibit similar representations. However, under NLOS conditions, the SWiT model demonstrates that it can clearly differentiate between users in various regions and determine if they are in LOS or NLOS. This is shown in Figure 10a and b, where each color denotes distinct spots.