Abstract This thesis mainly focuses on the analysis of the two-way massive MIMO amplify-and-forward (AF) relaying systems. Different from the prior works, all nodes are equipped with multiple antennas. Specifically, massive number of antennas and a small number of antennas are equipped at the relay and users, respectively. Due to limited space at each node, especially at the relay with massive antennas, spatial correlation among the multiple antennas generally occurs and would significantly affect the performance in practice, it is thus necessary to theoretically analyze the impact of antenna correlations on system performance for practical system design. However it is usually assumed not existence in most of the existing analyses for simplicity. To investigate the impact of the spatial correlation, the Kronecker channel model, which can characterize the antenna correlations at the transmitter and receiver sides separately is adopted in this thesis. Three widely applied two-way massive MIMO relaying systems are particularly investigated and meaningful insights are found as listed in the following. • A multi-pair full-duplex two-way relaying system, in which the relay adopts the popular maximum ratio combining/maximum ratio transmission (MRC/MRT) AF scheme, is first investigated. All users and the relay transmit and receive signals simultaneously due to the full-duplex mode. Power scaling law at both the users and the relay are mainly analyzed. The deterministic sum-rate under a general power scaling law is first derived. Based on this result, to obtain a non-vanishing sum-rate, four special power scaling cases are then discussed and the corresponding asymptotic sum-rate is given in simple form with clear insights. It is found that the transmit power at both the users and the relay can be scaled down inversely proportionally to the number of antennas at the relay while maintaining a desirable sum-rate with increasing number of antennas at the relay. Additionally, the impacts of the self-loop interference caused by full-duplex mode, intra-group interference and the antenna correlation are clearly quantified. The results show that the antenna correlation at users has complicated impact on the sum-rate while the antenna correlation at the relay could decrease the performance. • Single-pair full-duplex two-way massive MIMO AF relaying system with the zero- forcing (ZF) scheme at the relay is next investigated. The spatial correlation effect from the users and the relay is mainly analyzed with majorization theory. The results show that the asymptotic sum-rate is Schur-concave with respect to the eigenvalue vectors of the involved antenna correlation matrices at both the users’ transceiver sides and the relay’s receiver side, which means that the antenna correlations at both the users and the relay have detrimental effects on the sum-rate. Moreover, the antenna correlation at the users has more significant impact to the asymptotic sum-rate than that at the relay. • In the above two analyses, ideal hardware implementation is assumed in the transceivers of massive MIMO relaying systems. Since the hardware cost increases linearly with the number of antennas at each node, the cost becomes very high and unaffordable when the massive number of antennas are deployed. In practice, low cost/cheap hardware is desirable for the implementation of wireless networks. Hence, we take both this practical hardware impairments due to low cost implementation and antenna correlation into consideration and analyze their impacts on the performance of the half-duplex two-way massive MIMO AF relaying system. Joint scaling of transmission powers and hardware impairments is particularly investigated. It is shown that down scaling of the transmission powers at the users and the relay and up scaling of the hardware impairments at the relay with the number of relay antennas are tolerable without reducing the expected rate. However, user hardware impairment is a key limiting factor to the achievable rate and is not allowed to scale up with the number of relay antennas in order to achieve a non-vanishing rate. Moreover, ceiling effect on the achievable rate is observable. More interestingly, scalings of the user transmission power and the relay hardware impairment are found to be off-setable, which means that the relay hardware cost and the user transmission power are tradable. It is found that the best tradeoff is achieved in the medium scalings of both the relay hardware impairment and user transmission power.