contentlayer

ℹ️ [TLDR] Many smaller bug fixes, improvements and updated dependencies

Improvements

• Added aspectRatio property to ImageFieldData for type: image fields
• Added images example
• Support for esbuild 0.18 (closes #496)
• Upgraded various dependencies (incl. yaml package - closes #488)
• Added MIT license to all sub packages in the Contentlayer mono repo (closes #482)
• next-contentlayer should have contentlayer as peerDependency #447

Bug fixes

• Fix Bug in calculation of "_raw.flattenedPath" (closes #487)

ℹ️ [TLDR] New onSuccess callback that runs after completing a build successfully

✨ onSuccess Callback

A new callback will now be called when a successful build has completed.

The callback function receives a single argument that is an asynchronous function from which you can access data objects processed and generated by Contentlayer.

import { makeSource } from '@contentlayer/source-files'

export default makeSource({
  onSuccess: async (importData) => {
    const { allDocuments } = await importData()
    console.log('allDocuments', allDocuments.length)
  }
})

Running a build with the above configuration would yield something like the following on the console.

allDocuments 3

Closes #473

Better Non-Latin Character Support

Support has improved for characters in non-Latin languages. Fixes #337.

🙌 Thanks to @huanfe1 for help!

Other Improvements

Here are the other improvements shipped with this version.

Fix Body Field Handling for MDX

@stefanprobst resolved the discrepancy in handling a body field in frontmatter. Now, both Markdown and MDX files behave in the same way, supporting a body field in the frontmatter. See #451 for details.

Upgraded Dependencies

Dependencies have been upgraded to avoid warning messages. Fixes #360.

ℹ️ [TLDR] Bug fixes for next dev, Support for next export, bug fixes and updated dependencies

Improved next-contentlayer integration

As part of 0.3.2 we've overhauled the next-contentlayer integration with the goal of making it more stable and less dependent on implementation details of Next.js. This fixes #415 and #416 (thanks @kamto7).

As part of this effort (and by no longer relying on the redirects workaround) Contentlayer now also works with next export. (closes #426)

Other Improvements

• Fix: Opentelemetry version incompatibility with next 13.2 (closes #407 - thanks @jgillich)
• Fix: Type resolution when using modern TypeScript module resolution (closes #373 - thanks @jrolfs)
• Fix: Korean file names are not supported (closes #431 - thanks @mi-reu)
• Fix: contentDirInclude didn't work in some cases (closes #383 - thanks to @teobler)

Note about state of the project

Please also take a look at #429 to read about the current state of the project. 💜

ℹ️ [TLDR] React Server Components support, Dynamic content fetching (experimental), updated dependencies, bug fixes

React Server Components (RSC) support

We're super excited to announce that Contentlayer now supports React Server Components (RSC) out of the box! 🎉

We've updated our Next.js example to use RSC and it works like a charm. You can find the full example here. (Our docs will be updated shortly as well.)

We now recommend using RSC over the old getStaticProps/getStaticPaths approach. RSC is much more flexible and even allows you to use Contentlayer's dynamic content fetching API (see below).

Note: While it's theoretically also possible to use Contentlayer combined with the 'use client' approach, we don't recommend it as it massively increases page sizes and thus the page load time.

Experimental: Dynamic content fetching (e.g. in React Server Components)

Contentlayer is mostly used to build content-based static sites. However, in some cases it can be required/useful to fetch & process (remote) content dynamically at runtime (e.g. via React Server Components). This is now possible with the new (still experimental) fetchContent API for the contentlayer/source-remote-files content source. (Closes #85).

Here is a shortend example of how to use it (see full example for full details):

// app/some-dynamic-page.tsx
import { fetchContent } from 'contentlayer/generated'

export default function SomeDynamicPage({ }) {
  const contentResult = await fetchContent('some-branch')

  return <div>{content}</div>
}

// contentlayer.config.ts
import { defineDocumentType } from 'contentlayer/source-files'
import { makeSource } from 'contentlayer/source-remote-files'

const Post = defineDocumentType(() => ({
  // ...
}))

const syncContentFromGit = async ({ contentDir, gitTag }: { contentDir: string; gitTag: string }) => {
  // See full example
}

export default makeSource((contentBranch = 'main') => ({
  syncFiles: (contentDir) => syncContentFromGit({ contentDir, gitTag: contentBranch }),
  contentDirPath: `content/repo-${sourceKey}`,
  documentTypes: [Post],
  experimental: { enableDynamicBuild: true },
  //              ^^^^^^^^^^^^^^^^^^ enable dynamic content fetching
}))

Other Improvements

• Fix: Unable to install contentlayer in NextJs v13.2.1 (closes #386)
• Fix: contentType data doesn't support empty files (closes #361)
• Fix: Replace faker with alternative library (closes #217 - thanks @feliskio)
• Fix: Incorrect type inference from Stackbit config (closes #363)

ℹ️ [TLDR] New experimental source and required peer dependency update.

⚠️ Breaking Change: Updated esbuild Dependency

0.3.0 requires use of esbuild 0.17.0. You may need to update peer dependencies if experiencing installation issues.

✨ New Source: Remote Files [experimental]

While still focused on content coming from files, you can begin to explore loading content from files not located in your repository.

This works by syncing content from a remote location into your local workspace, and then behaves similarly to the files source. Contentlayer provides the hook (via a syncFiles property) for syncing the files, but you must write the code that pulls the files in.

Here is simple example with a remote Git repo and documentation.

import { makeSource } from 'contentlayer/source-remote-files'

export default makeSource({
  syncFiles: () => syncContentFromGit(),
  contentDirPath: 'remote-content',
  documentTypes: [Post],
  disableImportAliasWarning: true,
})

const syncContentFromGit = async () => {
  const syncRun = async () => {
    const repoAlreadyCloned = false
    if (repoAlreadyCloned) {
      // TODO `git clone` the repo
    } else {
      // TODO `git pull` the repo
    }
  }

  let wasCancelled = false
  let syncInterval

  const syncLoop = async () => {
    await syncRun()

    if (wasCancelled) return

    syncInterval = setTimeout(syncLoop, 1000 * 60)
  }

  syncLoop()

  return () => {
    wasCancelled = true
    clearTimeout(syncInterval)
  }
}

✨ New helper functions: defineComputedFields & defineFields

You can now use a defineComputedFields function to leverage the document type, including its static fields. Here's an example:

import { defineDocumentType, defineComputedFields } from 'contentlayer/source-files'

const computedFields = defineComputedFields<'Post'>({
  upperTitle: {
    type: 'string',
    resolve: (doc) => doc.title.toUpperCase(),
  },
})

const Post = defineDocumentType(() => ({
  name: 'Post',
  filePathPattern: `**/*.md`,
  fields: {
    // ...
  },
  computedFields,
}))

Other Improvements

• mdxOptions now always applies default Contentlayer remark plugins.
• Fixed a bug that avoids the issue demonstrated in #306.
• Upgraded dependencies.

Changes

Next.js 13 Support

Slightly delayed (sorry about that) Contentlayer now finally supports the Next.js version 13. Things should work just as they did before when using getStaticProps. 🚀

However, unfortunately React Server Components (RSC) can't yet be used with Contentlayer as there's a number of blocking bugs in Next.js itself (e.g. https://github.com/vercel/next.js/issues/41865) which need to be fixed first. You can track the progress here: https://github.com/contentlayerdev/contentlayer/issues/311

Other changes

• Improved types for useMDXComponent (https://github.com/contentlayerdev/contentlayer/pull/312 - thanks @Andrey-Bazhanov)
• Upgraded other deps

ℹ️ [TLDR] 0.2.8 improves list fields, field validations and error handling, type safety, and monorepo support.

✨ Improved Monorepo Support

When accessing documents outside the directory that contains contentlayer.config.ts, you can define contentDirPath using relative values. For example, consider a repo with the following directory structure:

.
├── docs [NextJS docs site]
└── components/
    ├── component-1/
    │   └──  README.md
    ├── component-2/
    │   └──  README.md
    └── component-3/
        └──  README.md

You can define define contentDirPath in docs/contentlayer.config.ts as .., allowing access to markdown files in the components directory.

export default makeSource({ 
  // ...
  contentDirPath: ".."
})

You can then run contentlayer build directly from the project subdirectory (docs in this example). See #295 for more details.

Avoiding INIT_CWD

This release also brings better support for various monorepo tooling — Lerna, Turborepo, NPM workspaces, etc. See #104 for details.

More list Field Types

#87 identified an issue with validating list field values. These validations have been improved, along with additional types within list fields. For example, previously, numbers were not supported, but are now available.

Type Safety for date Values

Being that there is no concept of a date type in JSON, Contentlayer stores date values as strings. Previously, these values were assigned a string type by Contentlayer's automatically-exported type definitions. Now the type is called IsoDateTimeString. It is an alias to string, but will make it easier to introduce type-safety for date fields in your projects.

export type Page = {
  // ...
  date: IsoDateTimeString
}

Other Improvements

• When encountering issues with singleton documents, Contentlayer will fail gracefully, with improved error messaging.
• Upgraded dependencies.

ℹ️ [TLDR] 0.2.7 brings experimental support for an image field when using files source.

✨ (Experimental) Support for image Field with Files Source

When using source-files as the content source, you can now use an image field. This will process images that are colocated with your content files, resolving the image path to a rich object containing properties for the image file. This will also move the image into a path that will ensure the image is publicly available on your site.

image Field Example (Files Source)

Given a document type definition that specifies a coverImage field of type image:

const Post = defineDocumentType(() => ({
  name: 'Post',
  filePathPattern: 'posts/*.md',
  fields: {
    coverImage: { type: 'image' },
  },
}))

And a content file that references a colocated image file:

---
coverImage: ./image-a.png
---

# Hello world

Contentlayer will produce the following for the coverImage property within the document:

"coverImage": {
  "filePath": "posts/image-a.png",
  "relativeFilePath": "image-a.png",
  "format": "png",
  "height": 480,
  "width": 640,
  "blurhashDataUrl": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAMAAADz0U65AAAACVBMVEV8Ou12OOBtM9E8a9LBAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAHElEQVQImWNgwAIYmZhgDEYwzcQEQiABRhDCAgADQQAWowgdtgAAAABJRU5ErkJggg=="
},

Date Improvements

date values that include timezones work more consistently (see #9 for details, and thanks to @pard68 & @mshick for their contributions).

This change removes the date-fns library in favor of the new Temporal API (via a polyfill).

Other Improvements

• There is now a resolveCwd when using the files content source to explicitly tell Contentlayer how to resolve the current working directory. This also changes the default resolution behavior. See #270 for the change, which closes #266. And thanks to @mshick for their contribution here.
• Upgraded dependencies.
• Strengthen codebase with more tests.

🐞 Bug Fixes

• Fix and better error logging when fields are missing from defineDocument. See #268 for details.

ℹ️ [TLDR] 0.2.6 contains some small improvements and a few bug fixes (e.g. when using PNPM).

Changes

• next-contentlayer: You can now set a custom configPath via createContentlayerPlugin in the Next.js plugin (similar to the --config CLI flag). See #248 for more - thanks to @stefanprobst for this nice contribution.
• next-contentlayer: Fixed a bug which caused "dev mode" to not work when using PNPM or when on Windows. Closes #239
• Fixed a peer dependency issue that resulted in an error when using PNPM. Closes #229.
• Fixed a TypeScript syntax problem for the generated index.d.ts file. Closes #253.
• source-files: Fixed a TypeScript definition for PartialArgs used in makeSource. Closes #243.

A special thanks to all contributors helping making this release happen. 💜

ℹ️ [TLDR] 0.2.5 brings significant flexibility to processing MDX and markdown documents, along with a number of smaller fixes and improvements.

✨ Markdown Processing

Contentlayer now supports custom processing for markdown content. This proposal was raised by @causztic in #202.

Previously, we were presenting developers with a base set of remark and rehype plugins for processing markdown. This prevented cases like being able to pass options to some of these plugins.

Rather than building out (opinionated) infrastructure to accommodate options for these base plugins, we chose to provide full flexibility in overriding these plugins and bringing your unified building pattern. This can be done via a markdown option passed to makeSource.

import rehypeStringify from 'rehype-stringify'
import remarkFrontmatter from 'remark-frontmatter'
import remarkParse from 'remark-parse'
import remark2rehype from 'remark-rehype'
 
makeSource({
  // your other options ...
  markdown: (builder) => {
    builder
      .use(remarkFrontmatter)
      .use(remarkParse)
      .use(remark2rehype)
      .use(rehypeStringify)
  }
})

⚠️ Please note the following:

• Take care to ensure that what you return from this function is an HTML string. We recommend you use rehypeStringify for this. Otherwise you may break Contentlayer's intended behavior.
• If using this markdown option, the remarkPlugins and rehypePlugins options will not be used. You should choose one approach tr the other.
• The code snippet above shows the default plugins used by Contentlayer. If you want to ensure compatibility, we recommend starting with these options.

✨ MDX Processing

To address #8 (from @mshick) and #192 (from @Saeris), we've added additional flexibility when processing mdx content. You can now pass mdxOptions as a makeSource option to modify the built-in MDX configuration, which is passed to the @mdx-js/mdx compile method.

makeSource({
  // your other options ...
  mdxOptions: { /* ... */ }
})

⚠️ Please note the following:

• If you use mdxOptions, both remarkPlugins and rehypePlugins options will be ignored. Choose one approach or the other.

Developer Experience Improvements

The following changes have been introduced to improve the developer experience:

✨ Contentlayer now makes sure you're using the same version of contentlayer and next-contentlayer and will provide a helpful error message if not. (#187 from studioprisoner)

Other Improvements & Fixes

🐞 _index.json still gets created when there are no content source files. (#208 from @jpedroschmitz)
🔧 Updated dependencies to support earlier versions of esbuild. (#204, #205 from @nayaabkhan)
🔧 Also updated various dependencies.
🔧 Avoid fatal error message (it wasn't a fatal error) during installation on Vercel. (#178)

🚀 Successful Beta Launch!

We officially released Contentlayer into Beta last month (April 2022)! 🎉 Since the launch we've seen a continued increase in community activity, including new issues, pull requests, and ideas.

A big thanks is due both to everyone who has helped Contentlayer get to this major milestone, but also to our newest members who are taking an active role in the continued evolution of what we believe will be the best way for developers to work with content for the web.

0.2.5 Contributors

The following community members who contributed to this release:

• causztic
• jpedroschmitz
• mshick
• nayaabkhan
• Saeris
• schickling
• seancdavis
• studioprisoner

Big thanks for these contributions, and a thanks is due to all those who have contributed ideas, feedback, and support that has led to these changes.

Now, onto the next iteration! ⚡

Changes

• Fix JSON import assertions for Node 17 (and higher). See #153 for more.
• Improved error printing

Changes

• Improved error handling for invalid date values
• Upgraded dependencies
  • Use mdx-bundler@9 - Now used MDX 2 instead of XDM (via #175 thanks @browniefed)
  • Updated other internal dependencies

Changes

• Fix react/jsx-runtime import error due to breaking change from React 17 to React 18. (Hopefully fixes #162 for good 🤞)

P.S. We're gearing up to officially launch Contentlayer in beta soon. Stay tuned for a fancy new website with much improved documentation and interactive examples. 🚀

🚨 This release contains one (1) breaking change. See below for more information and instructions to bring your project up to date.

💥 Breaking Changes

Because most folks are using Contentlayer as a Next.js plugin and not "calling" before using it, the API for wrapping the Next.js config object export has changed. #140

⚠️ Note: This only affects the Next.js plugin.

⬆️ To upgrade, remove the () following withContentlayer in your Next configuration file:

// next.config.mjs

import { withContentlayer } from 'next-contentlayer'

// ✅  Do This                 ▼▼▼ no additional `()` needed
export default withContentlayer({
  // Your Next.js config...
})

// ❌  Not This                 ▼▼
export default withContentlayer()({
  // Your Next.js config...
})

✨ This also introduces a createContentlayerPlugin API which allows for providing some non-default Contentlayer configuration options:

import { createContentlayerPlugin } from 'next-contentlayer'

const withContentlayer = createContentlayerPlugin({
  // Additional Contentlayer config options
})

export default withContentlayer({
  // Your Next.js config...
})

Other Changes

✨ Include/Exclude Content Files

Previously, Contentlayer picked up all files matching the filePathPattern within the contentDirPath. You can now explicitly exclude or include paths within the contentDirPath. This is especially useful when using your project root (.) to target content. #122

Both are an array of strings, with the following conditions:

• They can be either files or directories.
• The paths need to be relative to contentDirPath or absolute.
• An empty array means that all files in contentDirPath will be included.
• Glob/wildcard patterns (e.g. using *) are not supported yet.

There are sensible defaults in place. See below for details.

contentDirInclude should be an array of paths that Contentlayer should include. Defaults to []. This is useful when you have content spread across multiple directories in your project. Here is an example that targets only the docs directory in your project:

export default makeSource({
  // ...
  contentDirPath: ".",
  contentDirInclude: ["docs"],
});

contentDirExclude is an array of paths Contentlayer should explicitly exclude. When you set this, it overrides the default, which is: ['node_modules', '.git', '.yarn', '.cache', '.next', '.contentlayer', 'package.json', 'tsconfig.json'].

This is useful when you want to ignore a specific file or directory within your main content directory.

export default makeSource({
  // ...
  contentDirPath: "./content",
  contentDirExclude: ["internal-docs"],
});

✨ Improved Generated Content Structure for better performance

The .contentlayer/generated structure has changed slightly. Previously, it looked like this:

.contentlayer/generated/
├── Page/
│   ├── index.md.json
│   ├── about.md.json
│   └── blog.md.json
├── allPages.mjs
├── index.d.ts
├── index.mjs
└── types.d.ts

The all*.mjs files have been adding in the type directory as both _index.mjs and _index.json. The .mjs is used when running next dev for live reloading capabilities, while the JSON file is used to speed up production builds.

The new structure looks like this:

.contentlayer/generated/
├── Page/
│   ├── _index.mjs <-- for dev
│   ├── _index.json <-- for build
│   ├── index.md.json
│   ├── about.md.json
│   └── blog.md.json
├── index.d.ts
├── index.mjs
└── types.d.ts

🏎 This has improved build performance significantly!

✨ ESM-compatible JSON File Imports

assert { type: 'json' } is required when importing JSON files using Node v16.14 or higher. However, it is invalid syntax when using an earlier version of Node.

These assertions are now automatically added to generated .mjs files based on the current version of Node. #153

✨ Improved Developer Experience

Several updates provide better feedback when running Contentlayer.

• You can now disable alias warnings by defining disableImportAliasWarning: true in your Contentlayer config.

• Show warnings when path or baseUrl are missing from tsconfig.json or jsconfig.json. #132

• Fix an issue with the output suggesting it's skipping when it was actually processing appropriately. #156

Additional Updates

✨ Add support for React 18. #162
✨ Updated dependencies
✨ Content will update when changing Contentlayer configuration. #99

Changes

• next-contentlayer: Set onDemandEntries.maxInactiveAge Next.js config to one hour by default in order to improve live-reloading behaviour for content changes. (closes #63)

Changes

• Fix exports and typesVersions entries in generated .contentlayer/package.json file

🚨 This release contains two (2) breaking changes. See Upgrading below for more information and instructions on brings your project up to date.

Changes

💥 [Breaking] Generated files are now placed in a .contentlayer/generated directory at the root of the project. See Updating below for details. #65 #113
💥 [Breaking] bodyType will be replaced by contentType. See Upgrading below for details.

✨ The full markdown (or MDX) file content (including the frontmatter) will be passed to remark/rehype plugins during process only for the main body of the file. #106
✨ Live-reloading in Next.js via the useLiveReload has been improved. Reloading no longer scrolls to the top of the page and fixes a bug mentioned in #109.
✨ Types are generated as part of the npm postinstall script. This ensures types will exist right after cloning the project. #114 (also provides a workaround for #118)
✨ Improve type generation. #69 #89
✨ Upgrade to MDX v2 (via mdx-bundler@8).

Upgrading

The following sections outline breaking changes introduced in this version. Follow the steps below to upgrade your project.

1. New Directory for Generated Files

The .contentlayer directory was previously symlinked to a directory within node modules that caused a number of problems. By generating directly to a .contentlayer/generated directory, we see the following benefits:

• Live reload of schema changes now work without having to restart TypeScript server in VS Code. #65
• Auto-import works.
• Symlinks are no longer needed.
• Easier process to learn for new users.

Upgrade steps:

1. Delete node_modules/.contentlayer folder and .contentlayer symlink.

2. Add the new directory to your tsconfig.json / jsconfig.json as shown below:

   {
     "compilerOptions": {
       "baseUrl": ".",
       "paths": {
         "contentlayer/generated": ["./.contentlayer/generated"]
   //    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       }
     },
     "include": ["next-env.d.ts", "**/*.tsx", "**/*.ts", ".contentlayer/generated"]
   //                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
   }
   

3. Rename import statements from .contentlayer/data and .contentlayer/types to contentlayer/generated

// old
import { allPosts } from ".contentlayer/data"
import { Post } from ".contentlayer/types"

// new
import { allPosts, Post } from "contentlayer/generated"

2. Processing Main Content of a markdown/MDX File

bodyType has been deprecated in favor of contentType. The new options for contentType are markdown or mdx. This determines how the main body content will be processed.

This fixes an issue in which a .md was (intentionally) using MDX code in its main body area. This is now possible by specifying mdx as the contentType in the document type definition.

Upgrade steps:

• If you weren't using bodyType in your document definitions, then you don't need to do anything.

• If you were using markdown or mdx as the bodyType for a document type definition, change bodyType to contentType.

  // old
  defineDocumentType(() => ({
    name: "Page",
    filePathPattern: `**/*.mdx`,
    bodyType: "mdx"
  })
  
  // new
  defineDocumentType(() => ({
    name: "Page",
    filePathPattern: `**/*.mdx`,
    contentType: "mdx"
  })
  

• If you were using bodyType and had it set to none, remove it. The body will be assumed to be an empty string if there is no content, but the property will always exist in the output document.

  // old
  defineDocumentType(() => ({
    name: "Page",
    filePathPattern: `**/*.md`,
    bodyType: "none"
  })
  
  // new
  defineDocumentType(() => ({
    name: "Page",
    filePathPattern: `**/*.md`
  })
  

Changes

• Fixed: Optional list should not require the field to be set in the frontmatter. (closes #95)
• Fixed ESM export mapping for next-contentlayer/hooks. (closes #48)
• Experimental support for embedding referenced documents by setting embedDocument: true in type: 'reference' field definitions (tracked via #86)

Changes

• Improved error handling for singleton document types (i.e. Contentlayer now shows a helpful error message if no document was found)
• Improved data validation for MD(X) frontmatter
• Updated underlying dependencies

Changes

• Fix problem with incorrectly linked .contentlayer folder leading to build errors (e.g. on Vercel). (closes #44)