Building classification is pivotal in various applications, such as urban planning, navigation, and campus management. This paper presents a novel approach for building classification using a transformer-based model. The proposed framework integrates convolutional neural networks and self-attention mechanisms, leveraging the capabilities of both local and non-local feature extraction. We have curated a diverse dataset by taking images of various buildings at the National Institute of Technology Rourkela (NITR) campus. To address challenges posed by diverse lighting conditions, we have included images of both day and night. Our dataset includes 1600 images in total, having 10 classes. Image enhancement techniques such as histogram equalization are employed to mitigate the effects of poor illumination. Experimental results demonstrate the superiority of our approach in achieving robust classification under varying lighting conditions. Additionally, a user-friendly graphical user interface (GUI) is developed, enabling building classification through image uploads. The GitHub repository for this paper will be available at https://github.com/santoshpanda1995/NITRBuilding-Classification.