Image-based multiclass categorization of meteorological situations is essential for many applications, such as weather forecasting, autonomous cars, and surveillance systems. Self-driving vehicles can adapt their behavior and make wise judgments depending on road conditions, visibility, and possible risks when they are equipped with accurate weather condition categorization. Weather categorization facilitates real-time notifications and enhanced security procedures by helping surveillance cameras identify unfavorable weather conditions. By adding visual input into forecasts, automatic weather categorization from photographs improves the precision of weather forecasting algorithms. The ability to recognize weather patterns in photos helps track climate change and its effects on the environment. In order to categorize weather conditions, traditional image processing methods using simple regression-based machine learning algorithms rely on manually created features and rule-based methods. But they frequently have trouble extrapolating over a wide range of intricate weather patterns, which limits their scalability and accuracy. This work suggests a two-stage machine learning methodology for weather condition categorization in order to overcome the shortcomings of the current techniques. To extract high-level characteristics from a pretrained model, use deep learning models. In order to reduce the requirement for a sizable, annotated dataset, this stage seeks to extract rich representations from the input photos. Utilizing the retrieved characteristics as a foundation, create a deep learning classifier to increase classification robustness and accuracy. Lastly, the sunny, rainy, snowy, and hazy classes are classified using the suggested deep learning model. It also improves the model's capacity for generalization and lowers the likelihood of overfitting.