Machine learning (ML) has entered materials science arena as a fast and accurate tool for materials property prediction. Such ML models expect materials to be represented in numeric format (vector or array), also known as the material representation or descriptor. Two types of material descriptors exists, one that encodes crystal structure details for and the other that only uses chemical composition information with the hope of representing hypothetical compositions and discovering new ones. Recently, graph neural networks (GNNs) have shown excellent performance in predicting materials properties, typically outperforming classical ML algorithms. In this project, I developed a novel GNN architecture (named Finder) that integrates self-attention mechanism for predicting the properies of materials using their chemical composition or crystal structure. The proposed model displayed better performance, faster convergence and better domain transferability compared to some other state-of-the-art GNNs on materials benchmark datasets. Finder is listed as a high performer on Matbench benchmark suite leaderboard maintained by Bekerley Lab, USA.