directus / directus/gatsby-source-directus
gatsby-source-directus fails to generate pages in Deferred Static Generation (DSG)
- Dominant language
- JavaScript
- Stars
- 11
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
### Preflight Checklist
- [X] I have completed all [Troubleshooting Steps](https://docs.directus.io/getting-started/support/#troubleshooting-steps).
- [X] I'm on [the latest version of Directus](https://github.com/directus/directus/releases).
- [X] There's [no other issue](https://github.com/directus/directus/issues?q=is%3Aissue) that already describes my problem.
### Describe the Bug
[Gatsby DSG](https://www.gatsbyjs.com/docs/how-to/rendering-options/using-deferred-static-generation/) fails using gatsby-source-directus.
I set up a directus with gatsby-source-directus. What do I want to achieve is to have some of my collections being built only at the first request, by using the deferred static generation.
### To Reproduce
### In gatsby-node.js
Here, in short, the `defer : true` should identify the products as DSG items. You can find the option at the end of this snippet
```javascript
exports.createPages = async ({ graphql, actions }) => {
const { createPage } = actions
await graphql(`
{
directus {
product(limit: -1, filter: {status: {_eq: "published"}}) {
id
status
product_code
}
}
}
`).then(result => {
const templatePaths = {
product: path.resolve(`./src/templates/product.js`),
};
products.forEach((node) => {
createPage({
path: `${path}/${node.product_code}`,
component: templatePaths.product,
context: {
language,
productID: parseInt(node.id),
},
/*
When set to true, it tells Gatsby to exclude the page from the build step and instead generate it during the first HTTP request:
See https://www.gatsbyjs.com/docs/reference/rendering-options/deferred-static-generation/
*/
defer: true, // <-- this is the option to set the DSG generation for certain contents
})
})
})
}
```
### In gatsby-config.js
`batch: true` in graphql options must be set to true, in order to work with the `defer: true` in gatsby-config.js.
```javascript
module.exports = {
plugins: [
{
resolve: '@directus/gatsby-source-directus',
options: {
url: `http://localhost:8055`,
auth: {
token: `${token}`,
},
dev: {
refresh: '5s'
},
//[Optional] - defines extra options to be passed into gatsby-source-graphql. Useful for advanced use cases
graphql: {
batch: true,
// See https://github.com/graphql/dataloader#new-dataloaderbatchloadfn--options
// for a full list of DataLoader options
dataLoaderOptions: {
maxBatchSize: 512,
},
}
},
},
],
};
```
### Errors Shown
Once the setup is completed, if you `build` everything, nothing fails, and every page is generated.
```shell
┌ src/templates/family.js
│ ├ D /prodotti/product_name
│ └ D ...355 more pages available
├ src/templates/product.js
│ ├ D /prodotti/ANOTHER-PRODUCT
│ └ D ...17875 more pages available
├ src/pages/404.js
│ ├ /en/404.html
│ └ ...7 more pages available
└ src/pages/index.js
├ /en/
└ ...3 more pages available
╭────────────────────────────────────────────────────────────────╮
│ │
│ (SSG) Generated at build time │
│ D (DSG) Deferred static generation - page generated at runtime │
│ ∞ (SSR) Server-side renders at runtime (uses getServerData) │
│ λ (Function) Gatsby function │
│ │
╰────────────────────────────────────────────────────────────────╯
✨ Done in 43.97s.
```
### BUT
Once you `serve` what's in the build, and you try to access at your i.e. localhost:9000/your-item, you get this:
http://localhost:9000/prodotti/my-product
`Internal server error.`
Which leads to this error:
```shell
Rendering html for "/prodotti/my-product" failed. nodeFetch is not a function
**TypeError: nodeFetch is not a function**
```
-----
### POSSIBLE FIX
I already investigated for a possible solution
First, i tried to use only the **gatsby-source-graphq**l plugin, i.e.
```javascript
resolve: "gatsby-source-graphql",
options: {
typeName: "Directus",
fieldName: "directus",
url: "http://localhost:8055/graphql",
headers: {
Authorization: `Bearer ${token}`,
},
batch: true,
fetchOptions: {},
dataLoaderOptions: {
maxBatchSize: 512,
},
},
},
```
This way, the pages are all correctly built as with gatsby-source-directus, and they are generated properly in deferred static generation, while on `serve`. **Only**, the graphql obviously does not contains all the necessary graphql items, such as dyrectus_system.
So I managed that the problem should be in the **gatsby-source-directus**.
If you [check at this answer](https://github.com/gatsbyjs/gatsby/issues/33906#issuecomment-968748611) you can find that this was a problem common to gatsby-source-graphql too.
Therefore, I tried to change /node_modules/@directus/gatsby-source-directus/node_modules/gatsby-source-graphql/fetch.js
like this:
```javascript
//BEFORE
// const nodeFetch = require(`node-fetch`); // this is passed to the Apollo Link
//AFTER
const nodeFetch = require(`node-fetch`).default
```
And now it somehow works. Pages are correctly generated in DSG **BUT** it is a dirty, quick not and resolutive working fix.
In example, with this line, gatsby images are not fetched. And there should be many other issues which I had not investigated yet
### What version of Directus are you using?
9.7.0
### What version of Node.js are you using?
16.14.0
### What database are you using?
MySql
### What browser are you using?
Chrome, Brave
### What operating system are you using?
Linux 5.10.76-linuxkit
### How are you deploying Directus?
Docker
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the DSG failure using the gatsby-node.js and gatsby-config.js setup, then inspect node_modules/@directus/gatsby-source-directus/node_modules/gatsby-source-graphql/fetch.js and its node-fetch import. Compare the behavior with gatsby-source-graphql and test both build and serve, including Gatsby image fetching. Done means DSG pages render successfully without the internal server error or regressions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, javascript, node.js
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100