Adding Dynamic Sitemap to Next.js 14

As the landscape of web development continues to progress, ensuring your online presence is primed for search engines is imperative for attaining visibility and triumph. One pivotal element of this optimization involves maintaining a meticulously organized sitemap that efficiently communicates your website's structure and content to search engine bots.

A sitemap serves as a catalog detailing all the pages within a website, furnishing pertinent details about each page, such as its latest update and frequency of alterations. This data proves invaluable to search engines during the process of crawling and indexing your website. When coupled with complementary features like a navigational index, a well-crafted sitemap can markedly enhance your website's SEO efficacy.

In this article, we'll delve into the process of integrating a dynamic sitemap into a Next.js website. Our objective is to establish a dynamic sitemap that seamlessly adjusts to reflect any additions or deletions of pages within the website. This method is especially advantageous for SEO endeavors, particularly for websites featuring frequently refreshed content, such as blogs.

Integrating a sitemap into the app directory

With the advent of Next.js version 13, a fresh "app" directory was introduced as a prospective substitute for the existing "pages" directory in the foreseeable future. This transition has led to the replacement of Server-side rendering (utilizing getServerSideProps) with Server Components. Consequently, in our scenario, we can no longer depend on getServerSideProps for rendering our sitemap page.

For Next.js 13.2 and earlier versions

Thankfully, in Next.js 13.2 and prior versions, we can resort to Route Handlers to replicate the same functionality. Route Handlers enable the creation of custom request handlers tailored for specific routes utilizing the Web Request and Response APIs. They act as a substitute for API Routes from the "pages" directory.

Upon delving deeper into the documentation, we ascertain that Route Handlers can also be utilized to deliver non-UI content, precisely what our sitemap entails. Initially, we need to relocate the sitemap.xml.js file into the app directory and rename it to route.js, adhering to the standard filename for defining Route Handlers. Additionally, we'll nest it within the sitemap.xml folder.

Below is a comparison outlining the structural disparities between the erstwhile "pages" and the novel "app" directory structures:

// Before
blog/
  pages/
    sitemap.xml.js


// After
blog/
  app/
    sitemap.xml/
      route.js

Following this, we'll adjust the route.js to establish a bespoke GET request handler for this route, crafting a personalized response containing the newly generated sitemap. Furthermore, we'll eliminate the getServerSideProps function and the SiteMap component, as they are now redundant.

import { BlogPost } from '@/@types/blog'
import { getAllPosts } from '@/utils/api/splitbeeApi'

const URL = 'https://fabra.dev'

function generateSiteMap(posts: BlogPost[]) {
    return `<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="https://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
      <loc>${URL}</loc>
      <lastmod>2024-02-24</lastmod>
      <changefreq>yearly</changefreq>
      <priority>0.5</priority>
    </url>
    <url>
      <loc>${URL}/about-me</loc>
      <lastmod>2024-02-24</lastmod>
      <changefreq>yearly</changefreq>
      <priority>0.5</priority>
    </url>
    <url>
      <loc>${URL}/projects</loc>
      <lastmod>2024-02-24</lastmod>
      <changefreq>yearly</changefreq>
      <priority>0.5</priority>
    </url>
    <url>
      <loc>${URL}/technologies</loc>
      <lastmod>2024-02-24</lastmod>
      <changefreq>yearly</changefreq>
      <priority>0.5</priority>
    </url>
    <url>
      <loc>${URL}/tools</loc>
      <lastmod>2024-02-24</lastmod>
      <changefreq>yearly</changefreq>
      <priority>0.5</priority>
    </url>
    <url>
      <loc>${URL}/blog</loc>
      <lastmod>2024-02-24</lastmod>
      <changefreq>yearly</changefreq>
      <priority>0.5</priority>
    </url>
     ${posts.map(({ slug,date }) => {
        return `
           <url>
                <loc>${`${URL}/blog/${slug}`}</loc>
                <lastmod>${date}</lastmod>
                <changefreq>weekly</changefreq>
                <priority>0.5</priority>
           </url>
         `
    }).join('')}
   </urlset>
 `
}

export async function GET() {
    const posts = await getAllPosts()
    const body = generateSiteMap(posts)

    return new Response(body, {
        status: 200,
        headers: {
            'Cache-control': 'public, s-maxage=86400, stale-while-revalidate',
            'content-type': 'application/xml',
        },
    })
}

Here, we incorporate s-maxage of 1 day and stale-while-revalidate directives. Further details about them are accessible on MDN.

For Next.js 13.3 and beyond

Next.js version 13.3 unveiled the file-based metadata API, streamlining the manipulation of page metadata through the exportation of a Metadata object. This not only simplifies the management of static and dynamic sitemaps but also eradicates the necessity for manual sitemap generation.

To leverage this functionality, we can develop a sitemap.js file tasked with handling the majority of the repetitive logic entailed in crafting the website's sitemap.

// Before
blog/
  app/
    sitemap.xml/
          route.js

// After
blog/
  app/
    sitemap.js

Within the sitemap.js file, our primary objective revolves around mapping the posts and static URLs to the properties of a Sitemap object, subsequently returning this object.

// app/sitemap.ts

import { getSortedPostsData, PostData } from "../lib/posts";

const URL = "https://fabra.dev";

interface SitemapEntry {
  url: string;
  lastModified: string;
}

export default async function sitemap(): Promise<Array<SitemapEntry>> {
  const posts: Array<SitemapEntry> = getSortedPostsData().map((post: PostData) => ({
    url: `${URL}/blog/${post.id}`,
    lastModified: post.date,
  }));

  const routes: Array<SitemapEntry> = ["", "/portfolio", "/blog"].map((route: string) => ({
    url: `${URL}${route}`,
    lastModified: new Date().toISOString(),
  }));

  return [...routes, ...posts];
}

Furthermore, we introduce the lastModified field to showcase the date of the most recent modification for each page. This data proves valuable for search engines during the process of crawling and indexing your website.

With these alterations implemented, accessing http://fabra.dev/sitemap.xml should now exhibit the updated sitemap, dynamically rendered from the "app" directory.

Synopsis

Incorporating a sitemap into a website constitutes a pivotal facet of search engine optimization, particularly for expansive websites. It aids search engines in comprehending the structure of a site, uncovering all its pages, and efficiently crawling them.

This discourse elucidated the process of integrating a dynamic sitemap into a Next.js website, automatically adjusting whenever new pages are added or removed. It delineated the creation of a dynamic sitemap for Next.js websites utilizing either the "pages" or "app" directory.

A dynamic sitemap outperforms a static counterpart, particularly for larger or more intricate websites. By applying the methodologies elucidated in this discourse, you can guarantee that your websites are easily detectable by search engines, thereby enhancing their overall SEO efficacy.

References and Resources:

• Automate Your Deployment Workflow: Continuously Deploying Next.js website to DigitalOcean using GitHub Actions

• Create an Interactive Table of Contents for a Next.js Blog with Remark

• Getting Started with Template Literals in JavaScript

• Google: Sitemaps overview

• MDN: Cache Control

• Next-sitemap package

• Next.js: API Routes documentation

• Next.js: Route handlers documentation

• Next.js: getServerSideProps documentation

• Next.js: new app directory

• Next.js: version 13.3

• Sitemaps protocol