Fetching Data

Fetching data is a crucial aspect of many modern web applications. It involves retrieving data from a source, typically a server or an API (Application Programming Interface), and using that data to update your application's state or display on the user interface. Fetching data allows your application to be dynamic, displaying up-to-date information from databases, other websites, or user inputs. In this section, we'll explore the fundamental skills and concepts needed to fetch data effectively, starting with understanding HTTP and CORS, followed by mastering important JavaScript skills related to asynchronous operations and error handling, and finally applying these concepts in a React context using the useEffect hook.

Web Development Skills

Web development skills form the bedrock of any application that interacts with the internet, making them a crucial addition to our curriculum. This is the first time we're including this section because these skills, such as understanding HTTP and CORS, while not specific to JavaScript, are foundational to effectively fetching data from APIs.

Understanding HTTP

HTTP, or Hypertext Transfer Protocol, is the protocol used for transferring data over the internet. It defines a set of rules for how messages should be formatted and transmitted, and what actions web servers and browsers should take in response to various commands. Understanding HTTP is key to web development because it's the protocol used when making requests to APIs and receiving responses.

• Request: An HTTP request is made when you try to access data from a server. This could be when you're loading a webpage, submitting a form, or making an API call from your application. Each request contains a method, a URL, headers, and sometimes, a body with additional information.

• Response: An HTTP response is what the server sends back after processing your request. The response includes a status code, which tells you whether the request was successful or not, headers with metadata, and usually, a body with the requested data or a message detailing any errors.

HTTP Methods

HTTP methods, also known as HTTP verbs, are commands that indicate the desired action to be performed on a given resource. Each method represents a different type of operation and is used in different scenarios in web development.

Here's a table with a summary of the most commonly used HTTP methods:

Please note that while HTML forms only support GET and POST methods, JavaScript can use all these methods, providing more flexibility when interacting with APIs or other resources.

HTTP Status Codes

HTTP status codes are three-digit codes returned by a server in response to an HTTP request made by a client. These codes indicate the result of the request, providing a quick way to understand if the request was successful, and if not, what kind of error occurred.

These status codes are part of the HTTP standard, and they are used universally in web development. Understanding them can help you debug issues when making HTTP requests.

HTTP Headers

HTTP headers are an integral part of HTTP requests and responses. They carry additional information about the request or the response, or about the object sent in the message body. Headers can include fields indicating the requested page, the server or client sending the request, the date of the request, the type of data returned, and more. Understanding HTTP headers is key to controlling and diagnosing how your HTTP requests and responses are handled.

Here are some common headers that are included in HTTP requests and responses:

Request Headers:

• Accept: Indicates the type of data the client is expecting in the response. For example, Accept: application/json indicates that the client is expecting a JSON response.
• Content-Type: Indicates the type of data sent in the request body. For example, Content-Type: application/json indicates that the request body contains JSON data.
• Authorization: Contains credentials to authenticate a user-agent with a server, usually after the server has responded with a 401 Unauthorized status.
• User-Agent: Contains information about the user-agent originating the request, such as the browser type, operating system, and software version.

Response Headers:

• Access-Control-Allow-Origin: Specifies which websites are allowed to view the response. This is used in CORS (Cross-Origin Resource Sharing) headers.
• Content-Type: The Media type of the body of the response, like application/json, text/html, etc.
• Set-Cookie: Sends a cookie from the server to the user's browser.
• WWW-Authenticate: Defines the authentication method that should be used to access a resource after a 401 Unauthorized status has been sent.

These are just a few examples. There are many more HTTP headers, and they provide a wide range of information about the request and response.

Cross-Origin Resource Sharing (CORS)

Cross-Origin Resource Sharing, or CORS, is a mechanism that allows resources on a web page to be requested from another domain outside the domain from which the first resource was served. This is a security feature that prevents malicious scripts from accessing data from other websites. For example, if you're on https://www.example.com, a script on that page can't make a request to https://www.google.com unless the server at https://www.google.com explicitly allows it.

From the point of view of a web application developer, CORS is a mechanism that allows you to make requests to APIs from your application. Without CORS, you wouldn't be able to fetch data from an API, and your application would be limited to only displaying data from your own server.

JS Skills

In order to fetch data from an API, you'll need to master some fundamental JavaScript skills. These include understanding asynchronous programming, error handling, and using the Fetch API.

Error Handling (try/catch)

In JavaScript, try/catch statements are used to catch and handle errors or exceptions that occur in code.

The try block contains the code that may potentially throw an exception. The catch block contains the code that will be executed if an error is thrown in the try block.

Here's a simple example:

try {
  nonExistentFunction();
} catch (error) {
  console.error("An error occurred:", error.message);
}

The catch block takes the thrown error object as a parameter (in this case, error) and logs a message to the console along with the error message.

Asychronous Programming

Asynchronous programming is a design pattern which ensures the non-blocking code execution. Unlike traditional synchronous programming, where code is executed sequentially from top-to-bottom, asynchronous programming allows a unit of work to run separately from the main application thread. When the work is complete, it notifies the main thread about its completion or failure, allowing other operations to continue in the meantime.

In JavaScript, asynchronous programming is commonly used for actions that are time-consuming and thus would block further execution of code, such as network requests, reading files, or interacting with databases. Without asynchronous programming, these actions would block the main thread, causing the application to freeze until the action is complete.

Callbacks

Callbacks are a fundamental way of handling asynchronous operations in JavaScript. A callback function is passed as an argument to another function and is executed after its parent function has finished execution.

Here's an example of a callback function:

function doSomething(callback) {
  console.log("Doing something...");
  callback();
}

The doSomething function takes a callback function as an argument and executes it after logging a message to the console.

Promises

A Promise in JavaScript represents a value that may not be available yet but will be resolved at some point in the future, or it might get rejected. Promises are an improvement on callbacks for handling asynchronous operations, providing a more concise, clean syntax and better error handling.

A Promise has three states:

• pending: The Promise's result hasn't yet been determined.
• fulfilled: The operation completed successfully.
• rejected: The operation failed.

Here's a short example of creating and using a Promise:

let promise = new Promise((resolve, reject) => {
  let condition = true; // This can be the result of any asynchronous operation
 
  if (condition) {
    resolve("Operation was successful.");
  } else {
    reject("Operation failed.");
  }
});
 
promise
  .then((message) => {
    console.log(message); // 'Operation was successful.'
  })
  .catch((error) => {
    console.log(error); // 'Operation failed.'
  });

In this example, we create a new Promise that checks a condition (which could be the result of an asynchronous operation). If the condition is true, we call resolve() with a success message. If it's false, we call reject() with a failure message.

The then() method is called when the Promise is fulfilled, and the catch() method is called when the Promise is rejected. Both methods take a function as an argument, which is executed with the value passed to resolve() or reject(). This is how we can handle the result of a Promise once it's been resolved or rejected.

Async/Await

The async and await keywords in JavaScript are used to work with promises in a more comfortable, less .then() cluttered manner. An async function can contain an await expression that pauses the execution of the function and waits for the passed Promise's resolution, then resumes the function's execution and returns the resolved value.

Here's the same example as above, but using async/await instead of .then() and .catch():

let promise = new Promise((resolve, reject) => {
  let condition = true; // This can be the result of any asynchronous operation
 
  if (condition) {
    resolve("Operation was successful.");
  } else {
    reject("Operation failed.");
  }
});
 
async function executeAsyncTask() {
  try {
    const message = await promise;
    console.log(message); // 'Operation was successful.'
  } catch (error) {
    console.log(error); // 'Operation failed.'
  }
}
 
executeAsyncTask();

In this rewritten example, the promise code remains the same. What changes is how we handle the promise.

We create a new async function called executeAsyncTask(). Inside this function, we use the await keyword to pause execution of the function until promise settles. If the promise is fulfilled, the fulfilled value is assigned to message, and if the promise is rejected, an error is thrown.

The try/catch block is used to handle both the fulfillment and rejection of the promise. If the promise is fulfilled, the message is logged to the console. If the promise is rejected, the catch block catches the error, and the error is logged to the console.

This async/await version is easier to read and understand, especially when dealing with multiple asynchronous operations.

Fetch API

he Fetch API is a modern interface in JavaScript for making HTTP requests to resources across the network. It's built into the global scope of JavaScript, meaning you can use it in the browser without importing any additional resources. The Fetch API provides a more powerful and flexible feature set compared to older APIs like XMLHttpRequest.

The Fetch API returns a Promise that resolves to the Response object representing the response to the request. This can be used with the async/await syntax for easier handling.

Here's an example of how to use the Fetch API to make a GET request:

async function getData() {
  try {
    const response = await fetch("https://jsonplaceholder.typicode.com/posts");
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.error("Error:", error);
  }
}
 
getData();

In this example, we use fetch to make a GET request to the URL. The fetch method returns a Promise, which we await. Once the Promise resolves, it gives us a response object, from which we can extract the JSON data using the json() method. This also returns a Promise, which we again await. Once this Promise resolves, we have the data we requested, which we log to the console.

Here's an example of how to use the Fetch API to make a POST request:

async function postData() {
  try {
    const response = await fetch("https://jsonplaceholder.typicode.com/posts", {
      method: "POST",
      body: JSON.stringify({
        title: "foo",
        body: "bar",
        userId: 1,
      }),
      headers: {
        "Content-type": "application/json; charset=UTF-8",
      },
    });
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.error("Error:", error);
  }
}
 
postData();

In this example, we're making a POST request to the same URL, but this time we're sending data along with the request. We use the method property to specify that this is a POST request, the body property to provide the data we're sending, and the headers property to set the content type of our request. As before, we use fetch to make the request, await the Promise it returns, extract the JSON data from the response object, await the Promise this returns, and then log the data to the console.

Recommended YouTube video on Async Await, Promises, and Fetch API:

React Skills

useEffect

React's useEffect hook allows you to perform side effects in function components. Side effects could be anything that affects something outside of the scope of the function being executed, such as data fetching.

The useEffect hook runs after every render by default. However, you can control when it runs by passing in a second argument, an array of dependencies. If the values in this array remain the same between renders, React will skip executing the effect.

• If you pass an empty array ([]), the useEffect hook will run the effect only once after the initial render, similar to the 'mounted' phase in the component lifecycle.
• If you pass values (e.g., props or state) in the array, the effect will run whenever any of these values change.

Using useEffect with Fetch

The useEffect hook can be combined with the Fetch API or other data fetching libraries to retrieve data when a component loads. This data can then be stored in the component's local state and used as needed.

Here's an example of a component that fetches data from an API using useEffect:

"use client";

import { useState, useEffect } from 'react';

export default function PublicAPIs() {
  const [apis, setApis] = useState([]);
  const [error, setError] = useState(null);

  async function fetchAPIs() {
    try {
      const response = await fetch('https://api.publicapis.org/entries');
      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`);
      }
      const data = await response.json();
      setApis(data.entries);
      setError(null);
    } catch (e) {
      setError(e.message);
    }
  }

  useEffect(() => {
    fetchAPIs();
  }, []); // Run the effect only once after the initial render

  if (error) return (
    <div>
      <h2>Public APIs:</h2>
      <p>{error}</p>
    </div>
  );
  
  return (
    <div>
      <h2>Public APIs:</h2>
      {apis.length > 0 ? (
        <ul>
          {apis.map((api, index) => (
            <li key={index}>
              <a href={api.Link}>
                {api.API}
              </a>
              : {api.Description}
            </li>
          ))}
        </ul>
      ) : (
        <p>Loading...</p>
      )}
    </div>
  );
}

In this component, the useEffect hook runs the fetchAPIs function once after the initial render. This function fetches a list of public APIs from the API and sets the apis state with the fetched data. The component then renders a list of API names, or a "Loading..." message if the data is not yet available. This example is "meta" because it uses an API to fetch a list of APIs!

It is possible to view the data returned by the API by navigating directly to the API URL in your browser. For example, if you navigate to https://api.publicapis.org/entries (opens in a new tab), you'll see the data returned by the API.

However, this is not always the case. Some APIs require authentication or have other restrictions that prevent you from viewing the data in your browser. In these cases, you can use a tool like Postman (opens in a new tab) or REST Client (opens in a new tab) to view the data.

🗒️ Summary

• 🌐 Fetching data from a source like an API is critical for modern web applications to display up-to-date information.
• 🛠 Understanding HTTP, CORS, and mastering JavaScript skills related to asynchronous operations and error handling are fundamental for effective data fetching.
• 🔄 HTTP (Hypertext Transfer Protocol) is the protocol used for transferring data over the internet, including making requests to APIs and receiving responses.
• 🚀 HTTP methods (GET, POST, PUT, DELETE, HEAD, PATCH) indicate the desired action to be performed on a given resource.
• 🚦 HTTP status codes are three-digit codes returned by a server in response to an HTTP request, indicating the result of the request.
• 📃 HTTP headers carry additional information about the request or response.
• 🛡 CORS (Cross-Origin Resource Sharing) is a mechanism that allows resources on a web page to be requested from another domain outside the domain from which the first resource was served.
• 🎯 JavaScript skills for data fetching include understanding asynchronous programming, error handling, and using the Fetch API.
• ⚛ In React, the useEffect hook allows performing side effects in function components, such as data fetching.

📚 Knowledge Check

🏃 Activity

In this exercise, we'll be building an email validation component in React using the Disify API.

This component will allow a user to enter an email address into an input field, and then it will validate the email address by interacting with the Disify API. The Disify API provides valuable information about an email address such as whether it's in a valid format and if it's a disposable email.

Disposable email addresses are temporary email addresses that are typically used for a short period of time to sign up for services or receive information, after which they become inactive or are discarded, helping to protect user's primary email from spam or unwanted communications. However, they can also be used for malicious purposes, such as creating fake accounts or sending spam. Services like Disify exist primarily to help protect the integrity of online services, platforms, and applications from abuse and to maintain the quality of user data. For example, if you're building a social media platform, you might want to prevent users from signing up with disposable email addresses to prevent spam or abuse.

The initial code provided gives you a basic setup for the component. It fetches data from the API and then displays the raw JSON response data to the user. However, we can provide a much better user experience by interpreting the data and giving the user a clear and friendly message about the validity and disposability of the entered email. The output should be one of the following:

• "This email is valid and not disposable."
• "This email is valid and disposable."
• "This email is not valid."

In addition, the component fails to handle errors. Add appropriate error handling.

"use client";

import { useState } from "react";

export default function EmailCheckComponent() {
  const [email, setEmail] = useState("");
  const [result, setResult] = useState("");

  const validateEmail = async () => {
    const response = await fetch(`https://www.disify.com/api/email/${email}`);
    const data = await response.json();
    
    setResult(JSON.stringify(data));
  };


  return (
    <div>
      <input
        className="text-black border-2" 
        type="text"
        value={email}
        onChange={(e) => setEmail(e.target.value)}
        placeholder="Enter your email"
      />
      <button onClick={validateEmail} className="bg-blue-200">Check</button>
      <p>{result}</p>
    </div>
  );
}

💡 Hints

✅ Solution

"use client";

import { useState } from "react";

export default function EmailCheckComponent() {
  const [email, setEmail] = useState("");
  const [result, setResult] = useState("");

  const validateEmail = async () => {
    try {
      const response = await fetch(`https://www.disify.com/api/email/${email}`);
      const data = await response.json();
      if (data.format) {
        if (data.disposable) {
          setResult("The email address is valid and disposable.");
        } else {
          setResult("The email address is valid and not disposable.");
        }
      } else {
        setResult("The email address is invalid.");
      }
    } catch (error) {
      console.error(error);
    }
  };

  return (
    <div>
      <input
        className="text-black border-2" 
        type="text"
        value={email}
        onChange={(e) => setEmail(e.target.value)}
        placeholder="Enter your email"
      />
      <button onClick={validateEmail} className="bg-blue-200">Check</button>
      <p>{result}</p>
    </div>
  );
}

🌐 Community Events App

Week 7 of the Community Events App is all about fetching data from an API. The app fetches data from the Open Meteo API to display real-time weather data for Calgary.

• Week 7 Code (opens in a new tab)
• Live deployment (opens in a new tab)

The Weather component fetches and displays real-time weather data for a specific location, in this case, Calgary. The key steps and parts of the code are as follows:

1. useEffect Hook: The useEffect hook is used to perform side effects in the component such as data fetching. It takes two parameters: a function where you perform your side effects, and a dependency array.

   useEffect(() => {
     loadWeather(); // side effect
   }, []); // <-- dependency array

   This component's useEffect hook fetches data from a weather API. The function passed to useEffect is an Immediately Invoked Function Expression (IIFE) that is an async function. This is because useEffect itself cannot be an async function, but it can contain one.

   The dependency array is empty [], which means this effect will run only once after the component's initial render. If there were dependencies in this array, the effect would run again whenever those dependencies change.

2. State: The component has one piece of state: weather. The weather state is initialized to null and is updated with the fetched data. The function is asynchronous because it uses await to wait for the data to be fetched before updating the state.

   async function loadWeather() {
     try {
       const data = await fetchWeatherData();
       setWeather(data);
     } catch (error) {
       console.error(error);
     }
   }

3. Fetching Data: fetchWeatherData() is responsible for making the fetch request to the weather API.

   async function fetchWeatherData() {
     const response = await fetch(
       `https://api.open-meteo.com/v1/forecast?latitude=51.064&longitude=-114.08&current=temperature_2m,is_day,precipitation,rain,showers,snowfall,cloudcover&timezone=auto`
     );
     const data = await response.json();
     return data;
   }

   The fetchWeatherData() function sends a request to the weather API with specific parameters for latitude, longitude, and other weather details. After the data is fetched, it is converted to a JavaScript object using response.json(). The data is then returned.

4. Rendering the Component: The fetched data is displayed in the return statement of the component. If the data hasn't been fetched yet (i.e., weather is still null), "unavailable" will be displayed.

   <p>
     Temperature:{" "}
     {temp !== null && temp !== undefined && tempUnit
       ? `${temp}${tempUnit}`
       : "unavailable"}
   </p>
   <p>
     Cloud cover:{" "}
     {cloudCover !== null && cloudCover !== undefined && cloudCoverUnit
       ? `${cloudCover}${cloudCoverUnit}`
       : "unavailable"}
   </p>

   Here, the temperature and cloud cover are only displayed if their values are not null or undefined, and if the corresponding units are available. This ensures that even if the temperature or cloud cover is 0, it is correctly displayed.

📖 Further Reading

• Asynchronous Programming (opens in a new tab)
• useEffect Hook (opens in a new tab)
• Fetch API (opens in a new tab)