diff --git a/nws.ipynb b/nws.ipynb new file mode 100644 index 0000000..acc75f3 --- /dev/null +++ b/nws.ipynb @@ -0,0 +1,161 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "provenance": [], + "authorship_tag": "ABX9TyP9h1JYyXGqgDUWkosdnJiY" + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + }, + "language_info": { + "name": "python" + } + }, + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Rivulet (Weather): National Weather Service (NWS) API\n", + "_by Danny Zheng_" + ], + "metadata": { + "id": "uk5kKtkuGCMt" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Purpose of this Notebook\n", + "\n", + "This notebook was developed as part of NSF Grant 2445609 to support accessing and processing public data for middle and high school classroom activities.It's written to be relatively accessible to beginners, but if you have not interacted with computational notebooks or Python before, you may find navigating this tool difficult. (Check out the Show Your Work project for a gentle introduction to computational notebooks for educators!)\n", + "\n", + "Our project is focused on supporting data analysis and mechanistic reasoning in science education. In other words, we want students to learn how data provides information about how scientific mechanisms work and how understanding scientific mechanisms can help them to explain and interpret patterns in data. This builds on a long history of research on complex systems and agent-based modeling, and more closely connects that work to current expansions of data analysis across subjects.\n", + "\n", + "This tool focuses on connecting to the National Weather Service (NWS) API to access public weather data. The goal is to gather meteorological data (such as precipitation, air temperature, etc.) that can be used to explain and interpret patterns in water quality data. This supports the project's focus on helping students learn how data provides information about *how scientific mechanisms work*." + ], + "metadata": { + "id": "nkK-3s4ZGd1o" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Part I: Setup and Location\n", + "\n", + "The NWS API is public and does not require a secret key. We will define our imports and location in one place to streamline the process. The NWS API uses a latitude/longitude point to find weather data." + ], + "metadata": { + "id": "yts9RE19Gkvo" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "UQqtHh3lFCFw" + }, + "outputs": [], + "source": [ + "# Install necessary libraries for the NWS API and mapping\n", + "!pip install requests folium\n", + "\n", + "# Import libraries\n", + "import requests\n", + "import pandas as pd\n", + "import folium #Used to map\n", + "\n", + "# The NWS API requires a User-Agent for identification, NOT a secret key.\n", + "# IMPORTANT: Replace the email below with your actual email.\n", + "USER_AGENT = \"(Rivulet Project, dazzy0130@gmail.com)\"\n", + "\n", + "# EDIT HERE: Define the target latitude and longitude for your region.\n", + "lat = 37.7749 # Example: San Francisco\n", + "lon = -122.4194" + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Part I: Setup and Location\n", + "\n", + "The NWS API is public and does not require a secret key. We will define our imports and location in one place to streamline the process. The NWS API uses a latitude/longitude point to find weather data." + ], + "metadata": { + "id": "zMoFA9-WGpOT" + } + }, + { + "cell_type": "code", + "source": [ + "# Map the location\n", + "map_center = [lat, lon]\n", + "\n", + "# Create a Folium map object\n", + "m = folium.Map(location=map_center, zoom_start=12)\n", + "\n", + "# Add a marker to the map for the exact point\n", + "folium.Marker(\n", + " location=[lat, lon],\n", + " tooltip=\"Target NWS Location\"\n", + ").add_to(m)\n", + "\n", + "# Display the map\n", + "m" + ], + "metadata": { + "id": "-s2Trqs6FEx7" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "## Part III: Establishing the Connection\n", + "\n", + "This connection request is the foundational step. It retrieves the necessary metadata (the weather **grid ID** and **forecast URL**) that will be used for all subsequent weather data requests." + ], + "metadata": { + "id": "yeZunmhFGtPh" + } + }, + { + "cell_type": "code", + "source": [ + "# This API endpoint finds the correct weather grid for a given coordinate\n", + "api_endpoint = f\"https://api.weather.gov/points/{lat},{lon}\"\n", + "\n", + "# Set up the headers with our User-Agent (defined in Cell 6)\n", + "# Note: The NWS API requires this User-Agent header.\n", + "headers = {\n", + " 'User-Agent': USER_AGENT\n", + "}\n", + "\n", + "# 1. Make the request to the NWS API\n", + "response = requests.get(api_endpoint, headers=headers)\n", + "\n", + "# 2. Convert the successful JSON response into a Python dictionary\n", + "api_data = response.json()\n", + "\n", + "# The final output of the cell is the data dictionary, showing the key properties.\n", + "# Future steps will extract the URLs (like 'forecast') from this dictionary.\n", + "api_data['properties']" + ], + "metadata": { + "id": "Paht-fY4FR7y" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [], + "metadata": { + "id": "zsOy8aPiFTzx" + } + } + ] +} \ No newline at end of file diff --git a/stub.ipynb b/stub.ipynb new file mode 100644 index 0000000..aa98468 --- /dev/null +++ b/stub.ipynb @@ -0,0 +1,257 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "provenance": [], + "authorship_tag": "ABX9TyNtdKOicx69bgN3OtQmIM8x" + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + }, + "language_info": { + "name": "python" + } + }, + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Rivulet (Weather): National Weather Service (NWS) API\n", + "_by Danny Zheng_" + ], + "metadata": { + "id": "uk5kKtkuGCMt" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Purpose of this Notebook\n", + "\n", + "This notebook was developed as part of NSF Grant 2445609 to support accessing and processing public data for middle and high school classroom activities. It's written to be relatively accessible to beginners, but if you have not interacted with computational notebooks or Python before, you may find navigating this tool difficult. (Check out the Show Your Work project for a gentle introduction to computational notebooks for educators!)\n", + "\n", + "Our project is focused on supporting data analysis and mechanistic reasoning in science education. In other words, we want students to learn how data provides information about how scientific mechanisms work and how understanding scientific mechanisms can help them to explain and interpret patterns in data. This builds on a long history of research on complex systems and agent-based modeling, and more closely connects that work to current expansions of data analysis across subjects.\n", + "\n", + "This tool focuses on connecting to the National Weather Service (NWS) API to access public weather data. The goal is to gather meteorological data (such as precipitation, air temperature, etc.) that can be used to explain and interpret patterns in water quality data. This supports the project's focus on helping students learn how data provides information about *how scientific mechanisms work*." + ], + "metadata": { + "id": "nkK-3s4ZGd1o" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Part I: Setup and Location\n", + "\n", + "The NWS API is public and does not require a secret key. We will define our imports and location in one place to streamline the process. The NWS API uses a latitude/longitude point to find weather data." + ], + "metadata": { + "id": "yts9RE19Gkvo" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "UQqtHh3lFCFw" + }, + "outputs": [], + "source": [ + "# Install necessary libraries for the NWS API and mapping\n", + "!pip install requests folium\n", + "\n", + "# Import libraries\n", + "import requests\n", + "import pandas as pd\n", + "import folium #Used to map\n", + "\n", + "# The NWS API requires a User-Agent for identification, NOT a secret key.\n", + "# IMPORTANT: Replace the email below with your actual email.\n", + "USER_AGENT = \"(Rivulet Project, dazzy0130@gmail.com)\"\n", + "\n", + "# EDIT HERE: Define the target latitude and longitude for your region.\n", + "lat = 37.6213 # San Francisco International Airport (SFO)\n", + "lon = -122.3896 # San Francisco International Airport (SFO)" + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Part II: Specifying a Location and Visualizing\n", + "\n", + "Since the NWS API queries a single point (latitude and longitude), we will map this point to confirm the exact location being requested." + ], + "metadata": { + "id": "zMoFA9-WGpOT" + } + }, + { + "cell_type": "code", + "source": [ + "# Map the location\n", + "map_center = [lat, lon]\n", + "\n", + "# Create a Folium map object\n", + "m = folium.Map(location=map_center, zoom_start=12)\n", + "\n", + "# Add a marker to the map for the exact point\n", + "folium.Marker(\n", + " location=[lat, lon],\n", + " tooltip=\"Target NWS Location\"\n", + ").add_to(m)\n", + "\n", + "# Display the map\n", + "m" + ], + "metadata": { + "id": "-s2Trqs6FEx7" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "## Part III: Establishing the Connection\n", + "\n", + "This connection request is the foundational step. It retrieves the necessary metadata (the weather **grid ID** and **forecast URL**) that will be used for all subsequent weather data requests." + ], + "metadata": { + "id": "yeZunmhFGtPh" + } + }, + { + "cell_type": "code", + "source": [ + "# This API endpoint finds the correct weather grid for a given coordinate\n", + "api_endpoint = f\"https://api.weather.gov/points/{lat},{lon}\"\n", + "\n", + "# Set up the headers with our User-Agent (defined in Cell 6)\n", + "# Note: The NWS API requires this User-Agent header.\n", + "headers = {\n", + " 'User-Agent': USER_AGENT\n", + "}\n", + "\n", + "# 1. Make the request to the NWS API\n", + "response = requests.get(api_endpoint, headers=headers)\n", + "\n", + "# 2. Convert the successful JSON response into a Python dictionary\n", + "api_data = response.json()\n", + "\n", + "# The final output of the cell is the data dictionary, showing the key properties.\n", + "# Future steps will extract the URLs (like 'forecast') from this dictionary.\n", + "api_data['properties']" + ], + "metadata": { + "id": "Paht-fY4FR7y" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "## Part IV: Wind Vector Transformation for Mechanistic Reasoning\n", + "\n", + "We need historical wind data that is structured for easy analysis. This requires two steps: first, finding the nearest observation station with historical records, and second, fetching that data and converting the raw wind speed and direction into u (East-West) and v (North-South) vector components. This transformation is necessary to analyze wind changes over time." + ], + "metadata": { + "id": "4Qx8vCJGoGry" + } + }, + { + "cell_type": "code", + "source": [ + "# Install and import numpy, which is necessary for vector math\n", + "!pip install -q numpy\n", + "import numpy as np\n", + "from datetime import datetime, timedelta" + ], + "metadata": { + "id": "EiHIB90roI8i" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "stations_url = api_data['properties']['observationStations']\n", + "\n", + "# Request the list of stations. Note: headers are already defined in Cell 6.\n", + "stations_response = requests.get(stations_url, headers=headers)\n", + "stations_response.raise_for_status()\n", + "\n", + "# Extract data and find a reliable station (e.g., index 1 often reports complete wind data).\n", + "stations_data = stations_response.json()\n", + "nearest_station_url = stations_data['features'][1]['properties']['@id']\n", + "\n", + "# The output of the cell is the station URL, which is used in the next step.\n", + "nearest_station_url" + ], + "metadata": { + "id": "5gmGYth1oL0S" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "# Define the required time format for the NWS API\n", + "TIME_FORMAT = \"%Y-%m-%dT%H:%M:%SZ\"\n", + "\n", + "# EDIT HERE: Define the time period (e.g., past 7 days)\n", + "end_time = datetime.now().strftime(TIME_FORMAT)\n", + "start_time = (datetime.now() - timedelta(days=7)).strftime(TIME_FORMAT)\n", + "\n", + "observation_endpoint = f\"{nearest_station_url}/observations?start={start_time}&end={end_time}\"\n", + "\n", + "# Request the actual observation data\n", + "observation_response = requests.get(observation_endpoint, headers=headers)\n", + "observation_response.raise_for_status()\n", + "\n", + "observations = observation_response.json()\n", + "\n", + "# --- Data Transformation and Vector Calculation ---\n", + "clean_observations = []\n", + "for obs in observations['features']:\n", + " props = obs['properties']\n", + "\n", + " # Only process records that contain both wind speed and direction\n", + " if props['windSpeed']['value'] is not None and props['windDirection']['value'] is not None:\n", + "\n", + " speed = props['windSpeed']['value']\n", + " direction = props['windDirection']['value']\n", + "\n", + " # Convert direction to radians for vector math\n", + " direction_rad = np.deg2rad(direction)\n", + "\n", + " # Calculate u (East-West) and v (North-South) components\n", + " u_component = -speed * np.sin(direction_rad)\n", + " v_component = -speed * np.cos(direction_rad)\n", + "\n", + " clean_observations.append({\n", + " 'timestamp': props['timestamp'],\n", + " 'wind_speed_m_s': speed,\n", + " 'wind_direction_deg': direction,\n", + " 'u_component_m_s': u_component,\n", + " 'v_component_m_s': v_component\n", + " })\n", + "\n", + "# Convert the results to a DataFrame\n", + "df_wind_data = pd.DataFrame(clean_observations)\n", + "\n", + "# Display the final, transformed DataFrame\n", + "df_wind_data[['timestamp', 'wind_speed_m_s', 'u_component_m_s', 'v_component_m_s']].head()" + ], + "metadata": { + "id": "wcM7BRnRoSGZ" + }, + "execution_count": null, + "outputs": [] + } + ] +} \ No newline at end of file