Date sitemap lastmod from the build pipeline; warn on skipped pages
- bunext build records when each page's client bundle hash last changed
in .bunext/page-lastmod.json (sitemap.lastmod_file to relocate). The
hash covers the page's whole import graph, so component edits re-date
every page that uses them. The sitemap uses these dates as lastmod
- Add sitemap.default_lastmod
- BREAKING: use_file_lastmod now defaults to false (file mtimes are reset
by checkouts); it is a fallback after the build record
- Warn when a page or __root.tsx is skipped because its default export
is not at the start of a line (e.g. export { Page as default })
- Replace leaking module mocks in tests (log, grab-root-file-path) with
spies and fixtures
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
@@ -62,6 +62,7 @@ API routes live in `src/pages/api/` and follow standard Bun `Request → Respons
|
|||||||
|
|
||||||
### Bundler artifact tracking
|
### Bundler artifact tracking
|
||||||
`BundlerCTXMap` (stored in `global.BUNDLER_CTX_MAP`) maps each page to its bundled output path, content hash, and entrypoint. In production this is serialized to `public/pages/map.json` and loaded at startup.
|
`BundlerCTXMap` (stored in `global.BUNDLER_CTX_MAP`) maps each page to its bundled output path, content hash, and entrypoint. In production this is serialized to `public/pages/map.json` and loaded at startup.
|
||||||
|
`bunext build` also updates `.bunext/page-lastmod.json` (`src/utils/page-lastmod-record.ts`): per page, a fingerprint of the bundle hash + CSS path and the build date it last changed. The sitemap uses these dates as `lastmod`.
|
||||||
|
|
||||||
### Config
|
### Config
|
||||||
Consumer projects define `bunext.config.ts` at their project root. The `BunextConfig` type fields: `distDir`, `assetsPrefix`, `origin`, `globalVars`, `port`, `development`.
|
Consumer projects define `bunext.config.ts` at their project root. The `BunextConfig` type fields: `distDir`, `assetsPrefix`, `origin`, `globalVars`, `port`, `development`.
|
||||||
|
|||||||
@@ -39,6 +39,7 @@ The goal is a framework that is:
|
|||||||
- [Sitemap](#sitemap)
|
- [Sitemap](#sitemap)
|
||||||
- [Page `sitemap` Export](#page-sitemap-export)
|
- [Page `sitemap` Export](#page-sitemap-export)
|
||||||
- [`bunext sitemap` Command](#bunext-sitemap-command)
|
- [`bunext sitemap` Command](#bunext-sitemap-command)
|
||||||
|
- [Last-Modified Dates](#last-modified-dates)
|
||||||
- [Large Sitemaps](#large-sitemaps)
|
- [Large Sitemaps](#large-sitemaps)
|
||||||
- [Generating at Runtime](#generating-at-runtime)
|
- [Generating at Runtime](#generating-at-runtime)
|
||||||
- [robots.txt](#robotstxt)
|
- [robots.txt](#robotstxt)
|
||||||
@@ -702,6 +703,23 @@ bunext sitemap --origin https://staging.example.com
|
|||||||
bunext sitemap --no-robots # sitemap only
|
bunext sitemap --no-robots # sitemap only
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Last-Modified Dates
|
||||||
|
|
||||||
|
`lastmod` comes from the build pipeline. On every `bunext build`, Bunext compares each page's client bundle hash with the previous build and records the build date for pages whose bundle changed in `.bunext/page-lastmod.json`. The hash covers the page's whole import graph, so editing a shared component re-dates every page that uses it, while untouched pages keep their date. Editing `__root.tsx` changes every page.
|
||||||
|
|
||||||
|
Each URL's `lastmod` is the first of:
|
||||||
|
|
||||||
|
1. The entry's own `lastmod` (from `additional_paths` or a page's `sitemap` export)
|
||||||
|
2. The page's date from the build record (also used for entries returned by that page's `sitemap` export)
|
||||||
|
3. The page file's modification time, if `use_file_lastmod: true`
|
||||||
|
4. `default_lastmod`
|
||||||
|
|
||||||
|
With no date, `<lastmod>` is left out. Run `bunext build` before `bunext sitemap`; the command warns when there is no record.
|
||||||
|
|
||||||
|
- **The record must persist between builds.** Without the previous record, every page is dated with the current build. Building on the server you deploy to keeps `.bunext/` between builds. For CI builds from a fresh checkout, set `sitemap.lastmod_file` to a path you cache between runs or commit (it is sorted JSON for small diffs).
|
||||||
|
- **Server data is not covered.** Content loaded in a server function (e.g. from a database) does not change the bundle, so set `lastmod` on those entries in the page's `sitemap` export.
|
||||||
|
- Upgrading esbuild or changing build options can change every hash once, re-dating all pages.
|
||||||
|
|
||||||
#### Large Sitemaps
|
#### Large Sitemaps
|
||||||
|
|
||||||
A sitemap file may hold at most 50,000 URLs and 50MB. When the sitemap has more than `chunk_size` URLs (default 10,000), `bunext sitemap` splits it into chunk files and makes `sitemap.xml` a sitemap index pointing to them:
|
A sitemap file may hold at most 50,000 URLs and 50MB. When the sitemap has more than `chunk_size` URLs (default 10,000), `bunext sitemap` splits it into chunk files and makes `sitemap.xml` a sitemap index pointing to them:
|
||||||
@@ -756,7 +774,9 @@ export default async function handler(req: Request) {
|
|||||||
| `origin` | `string` | Base URL for each `<loc>`. Defaults to `config.origin`, then `http://localhost:<port>`. |
|
| `origin` | `string` | Base URL for each `<loc>`. Defaults to `config.origin`, then `http://localhost:<port>`. |
|
||||||
| `exclude` | `(string \| RegExp)[]` | Exact paths, subtrees ending in `/*` (`/admin/*` also excludes `/admin`), or regexes tested on the path. |
|
| `exclude` | `(string \| RegExp)[]` | Exact paths, subtrees ending in `/*` (`/admin/*` also excludes `/admin`), or regexes tested on the path. |
|
||||||
| `additional_paths` | `entries \| () => entries \| Promise<entries>` | Extra paths or entries (`path`, `lastmod`, `changefreq`, `priority`), or a sync/async function returning them, e.g. for dynamic routes. Each entry is a `string` or `BunextSitemapEntry`. |
|
| `additional_paths` | `entries \| () => entries \| Promise<entries>` | Extra paths or entries (`path`, `lastmod`, `changefreq`, `priority`), or a sync/async function returning them, e.g. for dynamic routes. Each entry is a `string` or `BunextSitemapEntry`. |
|
||||||
| `use_file_lastmod` | `boolean` | Use each page file's modification time as `lastmod`. Default `true`. |
|
| `default_lastmod` | `Date \| string` | `lastmod` for entries with no other date. See [Last-Modified Dates](#last-modified-dates). |
|
||||||
|
| `lastmod_file` | `string` | Page date record written by `bunext build`. Default `.bunext/page-lastmod.json`. |
|
||||||
|
| `use_file_lastmod` | `boolean` | Fall back to each page file's modification time. Default `false`. |
|
||||||
| `trailing_slash` | `boolean` | Append `/` to every non-root path. Default `false`. |
|
| `trailing_slash` | `boolean` | Append `/` to every non-root path. Default `false`. |
|
||||||
| `default_changefreq` | `BunextSitemapChangeFreq` | `changefreq` for entries that do not set one. |
|
| `default_changefreq` | `BunextSitemapChangeFreq` | `changefreq` for entries that do not set one. |
|
||||||
| `default_priority` | `number` | `priority` (0.0 to 1.0) for entries that do not set one. |
|
| `default_priority` | `number` | `priority` (0.0 to 1.0) for entries that do not set one. |
|
||||||
|
|||||||
Vendored
+13
@@ -4,6 +4,7 @@ import grabDirNames from "../../utils/grab-dir-names";
|
|||||||
import { rmSync, writeFileSync } from "fs";
|
import { rmSync, writeFileSync } from "fs";
|
||||||
import bunextInit from "../../functions/bunext-init";
|
import bunextInit from "../../functions/bunext-init";
|
||||||
import { v4 } from "uuid";
|
import { v4 } from "uuid";
|
||||||
|
import { updatePageLastmodRecord } from "../../utils/page-lastmod-record";
|
||||||
const { HYDRATION_DST_DIR, BUNX_CWD_PAGES_REWRITE_DIR, BUNDLER_CTX_MAP_JSON, SSR_BUNDLER_CTX_MAP_JSON, BUNX_CWD_MODULE_CACHE_DIR, BUILD_INFO_JSON, } = grabDirNames();
|
const { HYDRATION_DST_DIR, BUNX_CWD_PAGES_REWRITE_DIR, BUNDLER_CTX_MAP_JSON, SSR_BUNDLER_CTX_MAP_JSON, BUNX_CWD_MODULE_CACHE_DIR, BUILD_INFO_JSON, } = grabDirNames();
|
||||||
export default function () {
|
export default function () {
|
||||||
return new Command("build")
|
return new Command("build")
|
||||||
@@ -39,6 +40,18 @@ export default function () {
|
|||||||
build_date_unix: Date.now(),
|
build_date_unix: Date.now(),
|
||||||
build_date: Date(),
|
build_date: Date(),
|
||||||
};
|
};
|
||||||
|
if (global.BUNEXT_BUNDLER_CTX_MAP) {
|
||||||
|
try {
|
||||||
|
updatePageLastmodRecord({
|
||||||
|
ctx_map: global.BUNEXT_BUNDLER_CTX_MAP,
|
||||||
|
build_date: new Date(build_data.build_date_unix),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
catch (error) {
|
||||||
|
// Only sitemap dates depend on this; never fail the build.
|
||||||
|
log.warn(`Could not update page lastmod record: ${error?.message || error}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
writeFileSync(BUILD_INFO_JSON, JSON.stringify(build_data, null, 4), "utf-8");
|
writeFileSync(BUILD_INFO_JSON, JSON.stringify(build_data, null, 4), "utf-8");
|
||||||
log.success("Modules Built Successfully!");
|
log.success("Modules Built Successfully!");
|
||||||
process.exit();
|
process.exit();
|
||||||
|
|||||||
Vendored
+5
@@ -6,6 +6,7 @@ import grabDirNames from "../../utils/grab-dir-names";
|
|||||||
import grabConfig from "../../functions/grab-config";
|
import grabConfig from "../../functions/grab-config";
|
||||||
import grabConstants from "../../utils/grab-constants";
|
import grabConstants from "../../utils/grab-constants";
|
||||||
import { generateSitemapFiles } from "../../utils/generate-sitemap";
|
import { generateSitemapFiles } from "../../utils/generate-sitemap";
|
||||||
|
import { grabPageLastmodFile } from "../../utils/page-lastmod-record";
|
||||||
import generateRobotsTxt, { RobotsTxtGeneratedMarker, } from "../../utils/generate-robots-txt";
|
import generateRobotsTxt, { RobotsTxtGeneratedMarker, } from "../../utils/generate-robots-txt";
|
||||||
export default function () {
|
export default function () {
|
||||||
return new Command("sitemap")
|
return new Command("sitemap")
|
||||||
@@ -18,6 +19,10 @@ export default function () {
|
|||||||
global.BUNEXT_CONFIG = (await grabConfig()) || {};
|
global.BUNEXT_CONFIG = (await grabConfig()) || {};
|
||||||
global.BUNEXT_CONSTANTS = grabConstants();
|
global.BUNEXT_CONSTANTS = grabConstants();
|
||||||
const { output, chunk_size, chunks_dir, ...sitemap_config } = global.BUNEXT_CONFIG.sitemap || {};
|
const { output, chunk_size, chunks_dir, ...sitemap_config } = global.BUNEXT_CONFIG.sitemap || {};
|
||||||
|
if (!existsSync(grabPageLastmodFile(sitemap_config.lastmod_file))) {
|
||||||
|
log.warn(`No page lastmod record found. Run \`bunext build\` first ` +
|
||||||
|
`so pages get lastmod dates.`);
|
||||||
|
}
|
||||||
let sitemap_file;
|
let sitemap_file;
|
||||||
try {
|
try {
|
||||||
const output_path = options.output || output;
|
const output_path = options.output || output;
|
||||||
|
|||||||
+44
-2
@@ -2,6 +2,8 @@ import { existsSync } from "fs";
|
|||||||
import grabDirNames from "../../utils/grab-dir-names";
|
import grabDirNames from "../../utils/grab-dir-names";
|
||||||
import grabConstants from "../../utils/grab-constants";
|
import grabConstants from "../../utils/grab-constants";
|
||||||
import grabRootFilePath from "../server/web-pages/grab-root-file-path";
|
import grabRootFilePath from "../server/web-pages/grab-root-file-path";
|
||||||
|
import grabRelativePath from "../../utils/grab-relative-path";
|
||||||
|
import { log } from "../../utils/log";
|
||||||
const { ROOT_DIR } = grabDirNames();
|
const { ROOT_DIR } = grabDirNames();
|
||||||
export default async function grabClientHydrationScript({ page_local_path, }) {
|
export default async function grabClientHydrationScript({ page_local_path, }) {
|
||||||
const { ClientRootElementIDName, ClientRootComponentWindowName, ClientWindowPagePropsName, } = grabConstants();
|
const { ClientRootElementIDName, ClientRootComponentWindowName, ClientWindowPagePropsName, } = grabConstants();
|
||||||
@@ -18,12 +20,19 @@ export default async function grabClientHydrationScript({ page_local_path, }) {
|
|||||||
return undefined;
|
return undefined;
|
||||||
}
|
}
|
||||||
const root_content = await Bun.file(root_file_path).text();
|
const root_content = await Bun.file(root_file_path).text();
|
||||||
if (!root_content.match(/^export default/m)) {
|
if (!hasBundlableDefaultExport({
|
||||||
|
file_path: root_file_path,
|
||||||
|
content: root_content,
|
||||||
|
is_root: true,
|
||||||
|
})) {
|
||||||
return undefined;
|
return undefined;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const page_content = await Bun.file(page_local_path).text();
|
const page_content = await Bun.file(page_local_path).text();
|
||||||
if (!page_content.match(/^export default/m)) {
|
if (!hasBundlableDefaultExport({
|
||||||
|
file_path: page_local_path,
|
||||||
|
content: page_content,
|
||||||
|
})) {
|
||||||
return undefined;
|
return undefined;
|
||||||
}
|
}
|
||||||
let txt = ``;
|
let txt = ``;
|
||||||
@@ -54,3 +63,36 @@ export default async function grabClientHydrationScript({ page_local_path, }) {
|
|||||||
txt += `}\n`;
|
txt += `}\n`;
|
||||||
return txt;
|
return txt;
|
||||||
}
|
}
|
||||||
|
// File path -> content hash of the last warning, so rebuilds don't repeat
|
||||||
|
// a warning until the file changes.
|
||||||
|
const warned_files = new Map();
|
||||||
|
/**
|
||||||
|
* # Check a page or root file can be bundled
|
||||||
|
* Files are only bundled when `export default` starts a line. Files with
|
||||||
|
* no default export at all (e.g. a page still being written) are skipped
|
||||||
|
* silently; this warns about the confusing case where one exists
|
||||||
|
* elsewhere, like `import X from "x"; export default X` on one line, or
|
||||||
|
* `export { Page as default }`.
|
||||||
|
*/
|
||||||
|
function hasBundlableDefaultExport({ file_path, content, is_root, }) {
|
||||||
|
if (/^export default/m.test(content)) {
|
||||||
|
warned_files.delete(file_path);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
const has_default_export = /\bexport\s+default\b/.test(content) ||
|
||||||
|
/\bas\s+default\b/.test(content);
|
||||||
|
if (!has_default_export) {
|
||||||
|
warned_files.delete(file_path);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
const content_hash = Bun.hash(content);
|
||||||
|
if (warned_files.get(file_path) === content_hash)
|
||||||
|
return false;
|
||||||
|
warned_files.set(file_path, content_hash);
|
||||||
|
log.warn(`${grabRelativePath(file_path)} was skipped: ` +
|
||||||
|
`\`export default\` must be at the start of a line. ` +
|
||||||
|
(is_root
|
||||||
|
? `No pages can be bundled until the root layout is fixed.`
|
||||||
|
: `This page will not be bundled.`));
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|||||||
Vendored
+25
-2
@@ -438,10 +438,21 @@ export type BunextGenerateSitemapParams = {
|
|||||||
*/
|
*/
|
||||||
additional_paths?: BunextSitemapAdditionalPaths | (() => BunextSitemapAdditionalPaths | Promise<BunextSitemapAdditionalPaths>);
|
additional_paths?: BunextSitemapAdditionalPaths | (() => BunextSitemapAdditionalPaths | Promise<BunextSitemapAdditionalPaths>);
|
||||||
/**
|
/**
|
||||||
* Use each page file's modification time as `lastmod`. Defaults to
|
* Fall back to each page file's modification time as `lastmod` when
|
||||||
* `true`.
|
* the build record has no date for it. Defaults to `false`: checkouts
|
||||||
|
* reset modification times, so they are rarely meaningful.
|
||||||
*/
|
*/
|
||||||
use_file_lastmod?: boolean;
|
use_file_lastmod?: boolean;
|
||||||
|
/**
|
||||||
|
* `lastmod` for entries that have no other date
|
||||||
|
*/
|
||||||
|
default_lastmod?: Date | string;
|
||||||
|
/**
|
||||||
|
* Page last-modified record written by `bunext build`, relative to the
|
||||||
|
* project root. Defaults to `.bunext/page-lastmod.json`. Point it at a
|
||||||
|
* cached or committed location when builds start from a fresh checkout.
|
||||||
|
*/
|
||||||
|
lastmod_file?: string;
|
||||||
/**
|
/**
|
||||||
* Append a trailing slash to every non-root path. Defaults to `false`.
|
* Append a trailing slash to every non-root path. Defaults to `false`.
|
||||||
*/
|
*/
|
||||||
@@ -508,3 +519,15 @@ export type BunextRobotsConfig = BunextGenerateRobotsTxtParams & {
|
|||||||
*/
|
*/
|
||||||
output?: string;
|
output?: string;
|
||||||
};
|
};
|
||||||
|
/**
|
||||||
|
* Written by `bunext build`. Keyed by page path relative to the project
|
||||||
|
* root. `lastmod` is the build date at which `fingerprint` last changed.
|
||||||
|
*/
|
||||||
|
export type BunextPageLastmodRecord = {
|
||||||
|
pages: {
|
||||||
|
[local_path: string]: {
|
||||||
|
fingerprint: string;
|
||||||
|
lastmod: string;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|||||||
Vendored
+18
-6
@@ -7,6 +7,7 @@ import grabOrigin from "./grab-origin";
|
|||||||
import grabRelativePath from "./grab-relative-path";
|
import grabRelativePath from "./grab-relative-path";
|
||||||
import isDevelopment from "./is-development";
|
import isDevelopment from "./is-development";
|
||||||
import { log } from "./log";
|
import { log } from "./log";
|
||||||
|
import { readPageLastmodRecord } from "./page-lastmod-record";
|
||||||
const ErrorPagePaths = ["/404", "/500"];
|
const ErrorPagePaths = ["/404", "/500"];
|
||||||
/**
|
/**
|
||||||
* Protocol limits for a single sitemap file
|
* Protocol limits for a single sitemap file
|
||||||
@@ -131,7 +132,8 @@ function parseLastmod(lastmod) {
|
|||||||
return isNaN(date.getTime()) ? undefined : date;
|
return isNaN(date.getTime()) ? undefined : date;
|
||||||
}
|
}
|
||||||
export async function grabSitemapEntries(params) {
|
export async function grabSitemapEntries(params) {
|
||||||
const use_file_lastmod = params?.use_file_lastmod ?? true;
|
const use_file_lastmod = params?.use_file_lastmod ?? false;
|
||||||
|
const lastmod_record = readPageLastmodRecord(params?.lastmod_file);
|
||||||
const entries = [];
|
const entries = [];
|
||||||
const pages = grabAllPages({ exclude_api: true });
|
const pages = grabAllPages({ exclude_api: true });
|
||||||
for (let i = 0; i < pages.length; i++) {
|
for (let i = 0; i < pages.length; i++) {
|
||||||
@@ -145,17 +147,26 @@ export async function grabSitemapEntries(params) {
|
|||||||
file_path: page.local_path,
|
file_path: page.local_path,
|
||||||
route: page_path,
|
route: page_path,
|
||||||
});
|
});
|
||||||
|
// When the page's client bundle last changed, per `bunext build`
|
||||||
|
const build_lastmod = lastmod_record?.pages[grabRelativePath(page.local_path)]?.lastmod;
|
||||||
if (page_sitemap_paths) {
|
if (page_sitemap_paths) {
|
||||||
entries.push(...page_sitemap_paths.map(toSitemapEntry));
|
for (const page_sitemap_path of page_sitemap_paths) {
|
||||||
|
const entry = toSitemapEntry(page_sitemap_path);
|
||||||
|
entries.push({
|
||||||
|
...entry,
|
||||||
|
lastmod: entry.lastmod || build_lastmod,
|
||||||
|
});
|
||||||
|
}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if (page.url_path.match(/\[/))
|
if (page.url_path.match(/\[/))
|
||||||
continue;
|
continue;
|
||||||
entries.push({
|
entries.push({
|
||||||
path: page_path,
|
path: page_path,
|
||||||
lastmod: use_file_lastmod
|
lastmod: build_lastmod ||
|
||||||
|
(use_file_lastmod
|
||||||
? statSync(page.local_path).mtime
|
? statSync(page.local_path).mtime
|
||||||
: undefined,
|
: undefined),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
const additional_paths = typeof params?.additional_paths == "function"
|
const additional_paths = typeof params?.additional_paths == "function"
|
||||||
@@ -172,10 +183,11 @@ export async function grabSitemapEntries(params) {
|
|||||||
continue;
|
continue;
|
||||||
seen.add(path);
|
seen.add(path);
|
||||||
final_entries.push({
|
final_entries.push({
|
||||||
changefreq: params?.default_changefreq,
|
|
||||||
priority: params?.default_priority,
|
|
||||||
...entry,
|
...entry,
|
||||||
path,
|
path,
|
||||||
|
lastmod: entry.lastmod || params?.default_lastmod,
|
||||||
|
changefreq: entry.changefreq || params?.default_changefreq,
|
||||||
|
priority: entry.priority ?? params?.default_priority,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
return final_entries;
|
return final_entries;
|
||||||
|
|||||||
Vendored
+1
@@ -29,5 +29,6 @@ export type DirNames = {
|
|||||||
BUNDLER_CTX_MAP_JSON: string;
|
BUNDLER_CTX_MAP_JSON: string;
|
||||||
SSR_BUNDLER_CTX_MAP_JSON: string;
|
SSR_BUNDLER_CTX_MAP_JSON: string;
|
||||||
BUILD_INFO_JSON: string;
|
BUILD_INFO_JSON: string;
|
||||||
|
PAGE_LASTMOD_JSON: string;
|
||||||
};
|
};
|
||||||
export default function grabDirNames(): DirNames;
|
export default function grabDirNames(): DirNames;
|
||||||
|
|||||||
Vendored
+2
@@ -33,6 +33,7 @@ export default function grabDirNames() {
|
|||||||
const BUNDLER_CTX_MAP_JSON = path.join(BUNX_CWD_DIR, "bundler-ctx-map.json");
|
const BUNDLER_CTX_MAP_JSON = path.join(BUNX_CWD_DIR, "bundler-ctx-map.json");
|
||||||
const SSR_BUNDLER_CTX_MAP_JSON = path.join(BUNX_CWD_DIR, "ssr-bundler-ctx-map.json");
|
const SSR_BUNDLER_CTX_MAP_JSON = path.join(BUNX_CWD_DIR, "ssr-bundler-ctx-map.json");
|
||||||
const BUILD_INFO_JSON = path.join(BUNX_CWD_DIR, "build-info.json");
|
const BUILD_INFO_JSON = path.join(BUNX_CWD_DIR, "build-info.json");
|
||||||
|
const PAGE_LASTMOD_JSON = path.join(BUNX_CWD_DIR, "page-lastmod.json");
|
||||||
return {
|
return {
|
||||||
ROOT_DIR,
|
ROOT_DIR,
|
||||||
SRC_DIR,
|
SRC_DIR,
|
||||||
@@ -64,6 +65,7 @@ export default function grabDirNames() {
|
|||||||
BUNDLER_CTX_MAP_JSON,
|
BUNDLER_CTX_MAP_JSON,
|
||||||
SSR_BUNDLER_CTX_MAP_JSON,
|
SSR_BUNDLER_CTX_MAP_JSON,
|
||||||
BUILD_INFO_JSON,
|
BUILD_INFO_JSON,
|
||||||
|
PAGE_LASTMOD_JSON,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
// const NODE_MODULES_DIR = path.resolve(
|
// const NODE_MODULES_DIR = path.resolve(
|
||||||
|
|||||||
Vendored
+19
@@ -0,0 +1,19 @@
|
|||||||
|
import type { BundlerCTXMap, BunextPageLastmodRecord } from "../types";
|
||||||
|
/**
|
||||||
|
* # Path of the page last-modified record
|
||||||
|
*/
|
||||||
|
export declare function grabPageLastmodFile(lastmod_file?: string): string;
|
||||||
|
export declare function readPageLastmodRecord(lastmod_file?: string): BunextPageLastmodRecord | undefined;
|
||||||
|
/**
|
||||||
|
* # Record when each page's client bundle last changed
|
||||||
|
* A page's bundle hash covers its whole client import graph (including
|
||||||
|
* the root layout), so a component edit re-dates every page that uses it.
|
||||||
|
* Unchanged pages keep their previous date; removed pages are dropped.
|
||||||
|
*/
|
||||||
|
export declare function updatePageLastmodRecord({ ctx_map, build_date, lastmod_file, }: {
|
||||||
|
ctx_map: {
|
||||||
|
[local_path: string]: BundlerCTXMap;
|
||||||
|
};
|
||||||
|
build_date?: Date;
|
||||||
|
lastmod_file?: string;
|
||||||
|
}): BunextPageLastmodRecord;
|
||||||
Vendored
+51
@@ -0,0 +1,51 @@
|
|||||||
|
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
|
||||||
|
import path from "path";
|
||||||
|
import grabDirNames from "./grab-dir-names";
|
||||||
|
import { log } from "./log";
|
||||||
|
/**
|
||||||
|
* # Path of the page last-modified record
|
||||||
|
*/
|
||||||
|
export function grabPageLastmodFile(lastmod_file) {
|
||||||
|
const { ROOT_DIR, PAGE_LASTMOD_JSON } = grabDirNames();
|
||||||
|
const target = lastmod_file || global.BUNEXT_CONFIG?.sitemap?.lastmod_file;
|
||||||
|
return target ? path.resolve(ROOT_DIR, target) : PAGE_LASTMOD_JSON;
|
||||||
|
}
|
||||||
|
export function readPageLastmodRecord(lastmod_file) {
|
||||||
|
const file = grabPageLastmodFile(lastmod_file);
|
||||||
|
if (!existsSync(file))
|
||||||
|
return;
|
||||||
|
try {
|
||||||
|
const record = JSON.parse(readFileSync(file, "utf-8"));
|
||||||
|
if (record?.pages && typeof record.pages == "object")
|
||||||
|
return record;
|
||||||
|
}
|
||||||
|
catch { }
|
||||||
|
log.warn(`Ignoring invalid page lastmod record ${file}`);
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* # Record when each page's client bundle last changed
|
||||||
|
* A page's bundle hash covers its whole client import graph (including
|
||||||
|
* the root layout), so a component edit re-dates every page that uses it.
|
||||||
|
* Unchanged pages keep their previous date; removed pages are dropped.
|
||||||
|
*/
|
||||||
|
export function updatePageLastmodRecord({ ctx_map, build_date = new Date(), lastmod_file, }) {
|
||||||
|
const prior = readPageLastmodRecord(lastmod_file);
|
||||||
|
const record = { pages: {} };
|
||||||
|
// Sorted so a committed record produces minimal diffs
|
||||||
|
const local_paths = Object.keys(ctx_map).sort();
|
||||||
|
for (const local_path of local_paths) {
|
||||||
|
const artifact = ctx_map[local_path];
|
||||||
|
if (!artifact?.hash)
|
||||||
|
continue;
|
||||||
|
const fingerprint = `${artifact.hash}:${artifact.css_path || ""}`;
|
||||||
|
const prior_page = prior?.pages[local_path];
|
||||||
|
record.pages[local_path] =
|
||||||
|
prior_page?.fingerprint == fingerprint
|
||||||
|
? prior_page
|
||||||
|
: { fingerprint, lastmod: build_date.toISOString() };
|
||||||
|
}
|
||||||
|
const file = grabPageLastmodFile(lastmod_file);
|
||||||
|
mkdirSync(path.dirname(file), { recursive: true });
|
||||||
|
writeFileSync(file, JSON.stringify(record, null, 4), "utf-8");
|
||||||
|
return record;
|
||||||
|
}
|
||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@moduletrace/bunext",
|
"name": "@moduletrace/bunext",
|
||||||
"version": "1.1.22",
|
"version": "1.2.0",
|
||||||
"main": "dist/index.js",
|
"main": "dist/index.js",
|
||||||
"module": "index.ts",
|
"module": "index.ts",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
|||||||
@@ -525,4 +525,65 @@ describe("bunext sitemap", () => {
|
|||||||
expect(exists("public/sitemap.xml")).toBe(false);
|
expect(exists("public/sitemap.xml")).toBe(false);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("lastmod", () => {
|
||||||
|
const writeRecord = (file: string, pages: Record<string, string>) =>
|
||||||
|
writeProjectFile(
|
||||||
|
file,
|
||||||
|
JSON.stringify({
|
||||||
|
pages: Object.fromEntries(
|
||||||
|
Object.entries(pages).map(([page, lastmod]) => [
|
||||||
|
page,
|
||||||
|
{ fingerprint: "x", lastmod },
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
it("warns and omits lastmod when there is no build record", async () => {
|
||||||
|
writeConfig(`{ origin: "https://example.com" }`);
|
||||||
|
|
||||||
|
const res = await runSitemap();
|
||||||
|
|
||||||
|
expect(res.exit_code).toBe(0);
|
||||||
|
expect(res.output).toContain("Run `bunext build` first");
|
||||||
|
expect(readOutput()).not.toContain("<lastmod>");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("uses dates from .bunext/page-lastmod.json", async () => {
|
||||||
|
writeConfig(`{ origin: "https://example.com" }`);
|
||||||
|
writeRecord(".bunext/page-lastmod.json", {
|
||||||
|
"src/pages/about.tsx": "2026-05-01T00:00:00.000Z",
|
||||||
|
});
|
||||||
|
|
||||||
|
const res = await runSitemap();
|
||||||
|
const xml = readOutput();
|
||||||
|
|
||||||
|
expect(res.output).not.toContain("Run `bunext build` first");
|
||||||
|
expect(xml).toContain(
|
||||||
|
"<loc>https://example.com/about</loc>\n <lastmod>2026-05-01T00:00:00.000Z</lastmod>",
|
||||||
|
);
|
||||||
|
expect(xml).toContain(
|
||||||
|
"<loc>https://example.com/</loc>\n </url>",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reads the record from sitemap.lastmod_file", async () => {
|
||||||
|
writeConfig(`{
|
||||||
|
origin: "https://example.com",
|
||||||
|
sitemap: { lastmod_file: "seo/lastmod.json", default_lastmod: "2026-01-01" },
|
||||||
|
}`);
|
||||||
|
writeRecord("seo/lastmod.json", {
|
||||||
|
"src/pages/about.tsx": "2026-05-01T00:00:00.000Z",
|
||||||
|
});
|
||||||
|
|
||||||
|
await runSitemap();
|
||||||
|
const xml = readOutput();
|
||||||
|
|
||||||
|
expect(xml).toContain("<lastmod>2026-05-01T00:00:00.000Z</lastmod>");
|
||||||
|
expect(xml).toContain(
|
||||||
|
"<loc>https://example.com/</loc>\n <lastmod>2026-01-01</lastmod>",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,137 @@
|
|||||||
|
import {
|
||||||
|
describe,
|
||||||
|
it,
|
||||||
|
expect,
|
||||||
|
beforeAll,
|
||||||
|
beforeEach,
|
||||||
|
afterAll,
|
||||||
|
afterEach,
|
||||||
|
spyOn,
|
||||||
|
} from "bun:test";
|
||||||
|
import path from "path";
|
||||||
|
import fs from "fs";
|
||||||
|
import os from "os";
|
||||||
|
import grabClientHydrationScript from "../../../functions/bundler/grab-client-hydration-script";
|
||||||
|
import grabDirNames from "../../../utils/grab-dir-names";
|
||||||
|
import { log } from "../../../utils/log";
|
||||||
|
|
||||||
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "bunext-hydration-"));
|
||||||
|
const pagesDir = path.join(tmpDir, "pages");
|
||||||
|
|
||||||
|
let warnSpy: ReturnType<typeof spyOn>;
|
||||||
|
let fileCounter = 0;
|
||||||
|
|
||||||
|
/** Unique file per test so the once-per-file dedupe never leaks between tests */
|
||||||
|
function writePage(content: string, name = `page-${++fileCounter}.tsx`) {
|
||||||
|
const file_path = path.join(pagesDir, name);
|
||||||
|
fs.writeFileSync(file_path, content);
|
||||||
|
return file_path;
|
||||||
|
}
|
||||||
|
|
||||||
|
const warnings = () => warnSpy.mock.calls.map((c) => String(c[0]));
|
||||||
|
|
||||||
|
beforeAll(() => {
|
||||||
|
fs.mkdirSync(pagesDir, { recursive: true });
|
||||||
|
(global as any).BUNEXT_CONFIG = {};
|
||||||
|
(global as any).BUNEXT_DIR_NAMES = {
|
||||||
|
...grabDirNames(),
|
||||||
|
PAGES_DIR: pagesDir,
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
warnSpy = spyOn(log, "warn").mockImplementation(() => {});
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
warnSpy.mockRestore();
|
||||||
|
fs.rmSync(path.join(pagesDir, "__root.tsx"), { force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(() => {
|
||||||
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
|
(global as any).BUNEXT_DIR_NAMES = undefined;
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("grabClientHydrationScript default export warning", () => {
|
||||||
|
it("bundles a page with export default at the start of a line, without warning", async () => {
|
||||||
|
const page = writePage(
|
||||||
|
`import X from "x";\n\nexport default function Page() {\n return null;\n}\n`,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(await grabClientHydrationScript({ page_local_path: page })).toBeString();
|
||||||
|
expect(warnings()).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns when export default is not at the start of a line", async () => {
|
||||||
|
const page = writePage(
|
||||||
|
`import X from "x"; export default function Page() { return null; }\n`,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(await grabClientHydrationScript({ page_local_path: page })).toBeUndefined();
|
||||||
|
expect(warnings()).toHaveLength(1);
|
||||||
|
expect(warnings()[0]).toContain(path.basename(page));
|
||||||
|
expect(warnings()[0]).toContain("must be at the start of a line");
|
||||||
|
expect(warnings()[0]).toContain("This page will not be bundled");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns for `export { Page as default }`", async () => {
|
||||||
|
const page = writePage(
|
||||||
|
`function Page() {\n return null;\n}\n\nexport { Page as default };\n`,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(await grabClientHydrationScript({ page_local_path: page })).toBeUndefined();
|
||||||
|
expect(warnings()).toHaveLength(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("skips files with no default export silently", async () => {
|
||||||
|
const empty = writePage("");
|
||||||
|
const named = writePage(`export function Page() {\n return null;\n}\n`);
|
||||||
|
|
||||||
|
expect(await grabClientHydrationScript({ page_local_path: empty })).toBeUndefined();
|
||||||
|
expect(await grabClientHydrationScript({ page_local_path: named })).toBeUndefined();
|
||||||
|
expect(warnings()).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns once per file until its content changes", async () => {
|
||||||
|
const name = `page-repeat.tsx`;
|
||||||
|
const page = writePage(`const P = () => null; export default P;\n`, name);
|
||||||
|
|
||||||
|
await grabClientHydrationScript({ page_local_path: page });
|
||||||
|
await grabClientHydrationScript({ page_local_path: page });
|
||||||
|
expect(warnings()).toHaveLength(1);
|
||||||
|
|
||||||
|
writePage(`const P = () => <div />; export default P;\n`, name);
|
||||||
|
await grabClientHydrationScript({ page_local_path: page });
|
||||||
|
expect(warnings()).toHaveLength(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns again when a fixed file is broken the same way again", async () => {
|
||||||
|
const name = `page-flip.tsx`;
|
||||||
|
const broken = `const P = () => null; export default P;\n`;
|
||||||
|
const page = writePage(broken, name);
|
||||||
|
|
||||||
|
await grabClientHydrationScript({ page_local_path: page });
|
||||||
|
|
||||||
|
writePage(`const P = () => null;\nexport default P;\n`, name);
|
||||||
|
expect(await grabClientHydrationScript({ page_local_path: page })).toBeString();
|
||||||
|
|
||||||
|
writePage(broken, name);
|
||||||
|
await grabClientHydrationScript({ page_local_path: page });
|
||||||
|
|
||||||
|
expect(warnings()).toHaveLength(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns about a root layout with an unrecognized default export", async () => {
|
||||||
|
writePage(
|
||||||
|
`import type { PropsWithChildren } from "react"; export default function Root({ children }: PropsWithChildren) { return children; }\n`,
|
||||||
|
"__root.tsx",
|
||||||
|
);
|
||||||
|
const page = writePage(`export default function Page() {\n return null;\n}\n`);
|
||||||
|
|
||||||
|
expect(await grabClientHydrationScript({ page_local_path: page })).toBeUndefined();
|
||||||
|
expect(warnings()).toHaveLength(1);
|
||||||
|
expect(warnings()[0]).toContain("__root.tsx");
|
||||||
|
expect(warnings()[0]).toContain("No pages can be bundled");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -9,6 +9,7 @@ const { BUNX_TMP_DIR } = grabDirNames();
|
|||||||
|
|
||||||
describe("grabPageBundledReactComponent", () => {
|
describe("grabPageBundledReactComponent", () => {
|
||||||
const originalConfig = (globalThis as any).CONFIG;
|
const originalConfig = (globalThis as any).CONFIG;
|
||||||
|
const originalDirNames = global.BUNEXT_DIR_NAMES;
|
||||||
const fixtureDirs: string[] = [];
|
const fixtureDirs: string[] = [];
|
||||||
|
|
||||||
(globalThis as any).CONFIG = { development: true } as any;
|
(globalThis as any).CONFIG = { development: true } as any;
|
||||||
@@ -20,6 +21,7 @@ describe("grabPageBundledReactComponent", () => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
(globalThis as any).CONFIG = { development: true } as any;
|
(globalThis as any).CONFIG = { development: true } as any;
|
||||||
|
global.BUNEXT_DIR_NAMES = originalDirNames;
|
||||||
mock.restore();
|
mock.restore();
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -33,9 +35,16 @@ describe("grabPageBundledReactComponent", () => {
|
|||||||
const rootFilePath = path.join(fixtureDir, "__root.tsx");
|
const rootFilePath = path.join(fixtureDir, "__root.tsx");
|
||||||
const pageFilePath = path.join(fixtureDir, "page.tsx");
|
const pageFilePath = path.join(fixtureDir, "page.tsx");
|
||||||
|
|
||||||
mock.module("../../../../src/functions/server/web-pages/grab-root-file-path", () => ({
|
// Point the pages dir at the fixture instead of mocking
|
||||||
default: () => ({ root_file_path: rootFilePath })
|
// grab-root-file-path: module mocks leak into other test files
|
||||||
}));
|
// (mock.restore() does not undo mock.module).
|
||||||
|
const fs = require("fs");
|
||||||
|
fs.mkdirSync(fixtureDir, { recursive: true });
|
||||||
|
fs.writeFileSync(rootFilePath, "export default function Root() {}\n");
|
||||||
|
global.BUNEXT_DIR_NAMES = {
|
||||||
|
...grabDirNames(),
|
||||||
|
PAGES_DIR: fixtureDir,
|
||||||
|
};
|
||||||
|
|
||||||
mock.module("../../../../src/functions/server/web-pages/grab-tsx-string-module", () => ({
|
mock.module("../../../../src/functions/server/web-pages/grab-tsx-string-module", () => ({
|
||||||
default: async () => {
|
default: async () => {
|
||||||
|
|||||||
@@ -1,14 +1,7 @@
|
|||||||
import { afterEach, describe, expect, mock, test } from "bun:test";
|
import { afterEach, beforeEach, describe, expect, mock, spyOn, test } from "bun:test";
|
||||||
import path from "path";
|
import path from "path";
|
||||||
import type { GrabPageComponentRes } from "../../../types";
|
import type { GrabPageComponentRes } from "../../../types";
|
||||||
|
|
||||||
mock.module("../../../../src/utils/log", () => ({
|
|
||||||
log: {
|
|
||||||
info: mock((msg: string) => {}),
|
|
||||||
error: mock((msg: string) => {}),
|
|
||||||
},
|
|
||||||
}));
|
|
||||||
|
|
||||||
mock.module("../../../../src/utils/is-development", () => ({
|
mock.module("../../../../src/utils/is-development", () => ({
|
||||||
default: () => false
|
default: () => false
|
||||||
}));
|
}));
|
||||||
@@ -27,6 +20,13 @@ import grabPageComponent from "../../../../src/functions/server/web-pages/grab-p
|
|||||||
describe("grabPageComponent", () => {
|
describe("grabPageComponent", () => {
|
||||||
const originalRouter = global.BUNEXT_ROUTER;
|
const originalRouter = global.BUNEXT_ROUTER;
|
||||||
|
|
||||||
|
// Spy on the real logger rather than mocking the log module: module
|
||||||
|
// mocks leak into other test files and the mock lacked `warn`.
|
||||||
|
beforeEach(() => {
|
||||||
|
spyOn(log, "info").mockImplementation(() => {});
|
||||||
|
spyOn(log, "error").mockImplementation(() => {});
|
||||||
|
});
|
||||||
|
|
||||||
afterEach(() => {
|
afterEach(() => {
|
||||||
global.BUNEXT_ROUTER = originalRouter;
|
global.BUNEXT_ROUTER = originalRouter;
|
||||||
mock.restore();
|
mock.restore();
|
||||||
|
|||||||
@@ -1,16 +1,7 @@
|
|||||||
import { describe, expect, test, mock, afterEach, afterAll } from "bun:test";
|
import { describe, expect, test, mock, afterEach, afterAll, beforeEach, spyOn } from "bun:test";
|
||||||
import startServer from "../../../../src/functions/server/start-server";
|
import startServer from "../../../../src/functions/server/start-server";
|
||||||
import { log } from "../../../../src/utils/log";
|
import { log } from "../../../../src/utils/log";
|
||||||
|
|
||||||
// Mock log so we don't spam terminal during tests
|
|
||||||
mock.module("../../../../src/utils/log", () => ({
|
|
||||||
log: {
|
|
||||||
server: mock((msg: string) => {}),
|
|
||||||
info: mock((msg: string) => {}),
|
|
||||||
error: mock((msg: string) => {}),
|
|
||||||
}
|
|
||||||
}));
|
|
||||||
|
|
||||||
// Mock grabConfig so it doesn't try to look for bunext.config.ts and exit process
|
// Mock grabConfig so it doesn't try to look for bunext.config.ts and exit process
|
||||||
mock.module("../../../../src/functions/grab-config", () => ({
|
mock.module("../../../../src/functions/grab-config", () => ({
|
||||||
default: async () => ({})
|
default: async () => ({})
|
||||||
@@ -22,6 +13,14 @@ mock.module("../../../../src/utils/grab-app-port", () => ({
|
|||||||
}));
|
}));
|
||||||
|
|
||||||
describe("startServer", () => {
|
describe("startServer", () => {
|
||||||
|
// Silence the real logger with spies so we don't spam the terminal.
|
||||||
|
// Mocking the log module instead leaks into other test files.
|
||||||
|
beforeEach(() => {
|
||||||
|
spyOn(log, "server").mockImplementation(() => {});
|
||||||
|
spyOn(log, "info").mockImplementation(() => {});
|
||||||
|
spyOn(log, "error").mockImplementation(() => {});
|
||||||
|
});
|
||||||
|
|
||||||
afterEach(() => {
|
afterEach(() => {
|
||||||
if (global.BUNEXT_SERVER) {
|
if (global.BUNEXT_SERVER) {
|
||||||
global.BUNEXT_SERVER.stop(true);
|
global.BUNEXT_SERVER.stop(true);
|
||||||
@@ -30,8 +29,8 @@ describe("startServer", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
afterAll(() => {
|
afterAll(() => {
|
||||||
// Prevent the module mocks (log, grab-config, grab-app-port) from
|
// Restore the log spies. Note this does not undo the grab-config and
|
||||||
// leaking into other test files sharing this process.
|
// grab-app-port module mocks; mock.restore() only restores spies.
|
||||||
mock.restore();
|
mock.restore();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { describe, it, expect, beforeAll, afterAll } from "bun:test";
|
import { describe, it, expect, beforeAll, afterAll, afterEach } from "bun:test";
|
||||||
import path from "path";
|
import path from "path";
|
||||||
import fs from "fs";
|
import fs from "fs";
|
||||||
import os from "os";
|
import os from "os";
|
||||||
@@ -7,6 +7,7 @@ import generateSitemap, {
|
|||||||
grabSitemapEntries,
|
grabSitemapEntries,
|
||||||
SitemapMaxUrlsPerFile,
|
SitemapMaxUrlsPerFile,
|
||||||
} from "../../utils/generate-sitemap";
|
} from "../../utils/generate-sitemap";
|
||||||
|
import grabRelativePath from "../../utils/grab-relative-path";
|
||||||
|
|
||||||
// grabAllPages skips anything under __tests__, so fixtures live in the OS tmp dir
|
// grabAllPages skips anything under __tests__, so fixtures live in the OS tmp dir
|
||||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "bunext-sitemap-"));
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "bunext-sitemap-"));
|
||||||
@@ -120,7 +121,6 @@ describe("grabSitemapEntries", () => {
|
|||||||
expect(find("/products/2")?.priority).toBe(0.8);
|
expect(find("/products/2")?.priority).toBe(0.8);
|
||||||
expect(find("/contact")?.priority).toBe(0.3);
|
expect(find("/contact")?.priority).toBe(0.3);
|
||||||
expect(find("/contact")?.lastmod).toBeUndefined();
|
expect(find("/contact")?.lastmod).toBeUndefined();
|
||||||
expect(find("/about")?.lastmod).toBeInstanceOf(Date);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("accepts a sync or async function for additional paths", async () => {
|
it("accepts a sync or async function for additional paths", async () => {
|
||||||
@@ -145,12 +145,121 @@ describe("grabSitemapEntries", () => {
|
|||||||
expect(await paths({ trailing_slash: true })).toContain("/");
|
expect(await paths({ trailing_slash: true })).toContain("/");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("uses file mtime as lastmod unless disabled", async () => {
|
});
|
||||||
const [entry] = await grabSitemapEntries();
|
|
||||||
expect(entry.lastmod).toBeInstanceOf(Date);
|
|
||||||
|
|
||||||
const [no_lastmod] = await grabSitemapEntries({ use_file_lastmod: false });
|
describe("lastmod", () => {
|
||||||
expect(no_lastmod.lastmod).toBeUndefined();
|
const recordFile = path.join(tmpDir, "page-lastmod.json");
|
||||||
|
const pageKey = (file: string) =>
|
||||||
|
grabRelativePath(path.join(pagesDir, file));
|
||||||
|
|
||||||
|
const writeRecord = (pages: Record<string, string>) =>
|
||||||
|
fs.writeFileSync(
|
||||||
|
recordFile,
|
||||||
|
JSON.stringify({
|
||||||
|
pages: Object.fromEntries(
|
||||||
|
Object.entries(pages).map(([file, lastmod]) => [
|
||||||
|
pageKey(file),
|
||||||
|
{ fingerprint: "x", lastmod },
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
const lastmods = async (
|
||||||
|
params?: Parameters<typeof grabSitemapEntries>[0],
|
||||||
|
) =>
|
||||||
|
Object.fromEntries(
|
||||||
|
(
|
||||||
|
await grabSitemapEntries({ lastmod_file: recordFile, ...params })
|
||||||
|
).map((e) => [e.path, e.lastmod]),
|
||||||
|
);
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
fs.rmSync(recordFile, { force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("omits lastmod with no build record by default", async () => {
|
||||||
|
const result = await lastmods();
|
||||||
|
expect(result["/about"]).toBeUndefined();
|
||||||
|
expect(result["/"]).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("uses the build record date for static pages", async () => {
|
||||||
|
writeRecord({
|
||||||
|
"about.tsx": "2026-05-01T00:00:00.000Z",
|
||||||
|
"blog/index.tsx": "2026-06-01T00:00:00.000Z",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = await lastmods();
|
||||||
|
|
||||||
|
expect(result["/about"]).toBe("2026-05-01T00:00:00.000Z");
|
||||||
|
expect(result["/blog"]).toBe("2026-06-01T00:00:00.000Z");
|
||||||
|
expect(result["/"]).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("falls back to the page's build date for its sitemap export entries", async () => {
|
||||||
|
writeRecord({ "products/[id].tsx": "2026-04-01T00:00:00.000Z" });
|
||||||
|
|
||||||
|
const result = await lastmods({
|
||||||
|
additional_paths: [{ path: "/extra", lastmod: "2026-01-01" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result["/products/1"]).toBe("2026-04-01T00:00:00.000Z");
|
||||||
|
expect(result["/extra"]).toBe("2026-01-01");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("prefers an entry's own lastmod over the build record", async () => {
|
||||||
|
const datedDir = path.join(pagesDir, "dated");
|
||||||
|
fs.mkdirSync(datedDir, { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(datedDir, "[x].tsx"), "export default () => null");
|
||||||
|
fs.writeFileSync(
|
||||||
|
path.join(datedDir, "[x].server.ts"),
|
||||||
|
`export const sitemap = () => [
|
||||||
|
{ path: "/dated/own", lastmod: "2020-01-01" },
|
||||||
|
"/dated/none",
|
||||||
|
];`,
|
||||||
|
);
|
||||||
|
|
||||||
|
try {
|
||||||
|
writeRecord({ "dated/[x].tsx": "2026-04-01T00:00:00.000Z" });
|
||||||
|
|
||||||
|
const result = await lastmods();
|
||||||
|
|
||||||
|
expect(result["/dated/own"]).toBe("2020-01-01");
|
||||||
|
expect(result["/dated/none"]).toBe("2026-04-01T00:00:00.000Z");
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(datedDir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("uses default_lastmod for entries with no other date", async () => {
|
||||||
|
writeRecord({ "about.tsx": "2026-05-01T00:00:00.000Z" });
|
||||||
|
|
||||||
|
const result = await lastmods({
|
||||||
|
default_lastmod: "2026-09-09",
|
||||||
|
additional_paths: ["/extra"],
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result["/about"]).toBe("2026-05-01T00:00:00.000Z");
|
||||||
|
expect(result["/"]).toBe("2026-09-09");
|
||||||
|
expect(result["/extra"]).toBe("2026-09-09");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("falls back to file mtime only when use_file_lastmod is set", async () => {
|
||||||
|
writeRecord({ "about.tsx": "2026-05-01T00:00:00.000Z" });
|
||||||
|
|
||||||
|
const result = await lastmods({ use_file_lastmod: true });
|
||||||
|
|
||||||
|
expect(result["/about"]).toBe("2026-05-01T00:00:00.000Z");
|
||||||
|
expect(result["/"]).toBeInstanceOf(Date);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores an invalid record", async () => {
|
||||||
|
fs.writeFileSync(recordFile, "{ not json");
|
||||||
|
|
||||||
|
const result = await lastmods();
|
||||||
|
|
||||||
|
expect(result["/about"]).toBeUndefined();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,120 @@
|
|||||||
|
import { describe, it, expect, beforeEach, afterAll } from "bun:test";
|
||||||
|
import path from "path";
|
||||||
|
import fs from "fs";
|
||||||
|
import os from "os";
|
||||||
|
import type { BundlerCTXMap } from "../../types";
|
||||||
|
import {
|
||||||
|
grabPageLastmodFile,
|
||||||
|
readPageLastmodRecord,
|
||||||
|
updatePageLastmodRecord,
|
||||||
|
} from "../../utils/page-lastmod-record";
|
||||||
|
|
||||||
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "bunext-lastmod-"));
|
||||||
|
const recordFile = path.join(tmpDir, "nested", "page-lastmod.json");
|
||||||
|
|
||||||
|
const artifact = (hash: string, css_path?: string) =>
|
||||||
|
({ hash, css_path, path: `${hash}.js` }) as BundlerCTXMap;
|
||||||
|
|
||||||
|
const build = (
|
||||||
|
ctx_map: Record<string, BundlerCTXMap>,
|
||||||
|
build_date: string,
|
||||||
|
) =>
|
||||||
|
updatePageLastmodRecord({
|
||||||
|
ctx_map,
|
||||||
|
build_date: new Date(build_date),
|
||||||
|
lastmod_file: recordFile,
|
||||||
|
});
|
||||||
|
|
||||||
|
const day1 = "2026-01-01T00:00:00.000Z";
|
||||||
|
const day2 = "2026-02-01T00:00:00.000Z";
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
fs.rmSync(recordFile, { force: true });
|
||||||
|
(global as any).BUNEXT_CONFIG = {};
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(() => {
|
||||||
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("updatePageLastmodRecord", () => {
|
||||||
|
it("dates every page with the build date on the first build", () => {
|
||||||
|
const record = build(
|
||||||
|
{ "src/pages/a.tsx": artifact("h1"), "src/pages/b.tsx": artifact("h2") },
|
||||||
|
day1,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(record.pages["src/pages/a.tsx"].lastmod).toBe(day1);
|
||||||
|
expect(record.pages["src/pages/b.tsx"].lastmod).toBe(day1);
|
||||||
|
expect(readPageLastmodRecord(recordFile)).toEqual(record);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the date of unchanged pages and re-dates changed ones", () => {
|
||||||
|
build(
|
||||||
|
{ "src/pages/a.tsx": artifact("h1"), "src/pages/b.tsx": artifact("h2") },
|
||||||
|
day1,
|
||||||
|
);
|
||||||
|
|
||||||
|
const record = build(
|
||||||
|
{ "src/pages/a.tsx": artifact("h1"), "src/pages/b.tsx": artifact("h2-new") },
|
||||||
|
day2,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(record.pages["src/pages/a.tsx"].lastmod).toBe(day1);
|
||||||
|
expect(record.pages["src/pages/b.tsx"].lastmod).toBe(day2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("re-dates a page when only its CSS bundle changes", () => {
|
||||||
|
build({ "src/pages/a.tsx": artifact("h1", "a-1.css") }, day1);
|
||||||
|
const record = build({ "src/pages/a.tsx": artifact("h1", "a-2.css") }, day2);
|
||||||
|
|
||||||
|
expect(record.pages["src/pages/a.tsx"].lastmod).toBe(day2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops removed pages and dates new ones", () => {
|
||||||
|
build({ "src/pages/a.tsx": artifact("h1") }, day1);
|
||||||
|
const record = build({ "src/pages/new.tsx": artifact("h3") }, day2);
|
||||||
|
|
||||||
|
expect(Object.keys(record.pages)).toEqual(["src/pages/new.tsx"]);
|
||||||
|
expect(record.pages["src/pages/new.tsx"].lastmod).toBe(day2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("starts over from an invalid record", () => {
|
||||||
|
fs.mkdirSync(path.dirname(recordFile), { recursive: true });
|
||||||
|
fs.writeFileSync(recordFile, "nope");
|
||||||
|
|
||||||
|
const record = build({ "src/pages/a.tsx": artifact("h1") }, day2);
|
||||||
|
|
||||||
|
expect(record.pages["src/pages/a.tsx"].lastmod).toBe(day2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("writes pages in sorted order", () => {
|
||||||
|
build(
|
||||||
|
{ "src/pages/z.tsx": artifact("1"), "src/pages/a.tsx": artifact("2") },
|
||||||
|
day1,
|
||||||
|
);
|
||||||
|
|
||||||
|
const written = JSON.parse(fs.readFileSync(recordFile, "utf-8"));
|
||||||
|
expect(Object.keys(written.pages)).toEqual([
|
||||||
|
"src/pages/a.tsx",
|
||||||
|
"src/pages/z.tsx",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("grabPageLastmodFile", () => {
|
||||||
|
it("defaults to .bunext/page-lastmod.json", () => {
|
||||||
|
expect(grabPageLastmodFile()).toBe(
|
||||||
|
path.join(process.cwd(), ".bunext", "page-lastmod.json"),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("resolves sitemap.lastmod_file from config against the project root", () => {
|
||||||
|
(global as any).BUNEXT_CONFIG = {
|
||||||
|
sitemap: { lastmod_file: "seo/lastmod.json" },
|
||||||
|
};
|
||||||
|
expect(grabPageLastmodFile()).toBe(
|
||||||
|
path.join(process.cwd(), "seo", "lastmod.json"),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -5,6 +5,7 @@ import { rmSync, writeFileSync } from "fs";
|
|||||||
import bunextInit from "../../functions/bunext-init";
|
import bunextInit from "../../functions/bunext-init";
|
||||||
import type { BuildDataType } from "../../types";
|
import type { BuildDataType } from "../../types";
|
||||||
import { v4 } from "uuid";
|
import { v4 } from "uuid";
|
||||||
|
import { updatePageLastmodRecord } from "../../utils/page-lastmod-record";
|
||||||
|
|
||||||
const {
|
const {
|
||||||
HYDRATION_DST_DIR,
|
HYDRATION_DST_DIR,
|
||||||
@@ -69,6 +70,20 @@ export default function () {
|
|||||||
build_date: Date(),
|
build_date: Date(),
|
||||||
};
|
};
|
||||||
|
|
||||||
|
if (global.BUNEXT_BUNDLER_CTX_MAP) {
|
||||||
|
try {
|
||||||
|
updatePageLastmodRecord({
|
||||||
|
ctx_map: global.BUNEXT_BUNDLER_CTX_MAP,
|
||||||
|
build_date: new Date(build_data.build_date_unix),
|
||||||
|
});
|
||||||
|
} catch (error: any) {
|
||||||
|
// Only sitemap dates depend on this; never fail the build.
|
||||||
|
log.warn(
|
||||||
|
`Could not update page lastmod record: ${error?.message || error}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
BUILD_INFO_JSON,
|
BUILD_INFO_JSON,
|
||||||
JSON.stringify(build_data, null, 4),
|
JSON.stringify(build_data, null, 4),
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import grabDirNames from "../../utils/grab-dir-names";
|
|||||||
import grabConfig from "../../functions/grab-config";
|
import grabConfig from "../../functions/grab-config";
|
||||||
import grabConstants from "../../utils/grab-constants";
|
import grabConstants from "../../utils/grab-constants";
|
||||||
import { generateSitemapFiles } from "../../utils/generate-sitemap";
|
import { generateSitemapFiles } from "../../utils/generate-sitemap";
|
||||||
|
import { grabPageLastmodFile } from "../../utils/page-lastmod-record";
|
||||||
import generateRobotsTxt, {
|
import generateRobotsTxt, {
|
||||||
RobotsTxtGeneratedMarker,
|
RobotsTxtGeneratedMarker,
|
||||||
} from "../../utils/generate-robots-txt";
|
} from "../../utils/generate-robots-txt";
|
||||||
@@ -43,6 +44,13 @@ export default function () {
|
|||||||
const { output, chunk_size, chunks_dir, ...sitemap_config } =
|
const { output, chunk_size, chunks_dir, ...sitemap_config } =
|
||||||
global.BUNEXT_CONFIG.sitemap || {};
|
global.BUNEXT_CONFIG.sitemap || {};
|
||||||
|
|
||||||
|
if (!existsSync(grabPageLastmodFile(sitemap_config.lastmod_file))) {
|
||||||
|
log.warn(
|
||||||
|
`No page lastmod record found. Run \`bunext build\` first ` +
|
||||||
|
`so pages get lastmod dates.`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
let sitemap_file: string;
|
let sitemap_file: string;
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
|||||||
@@ -2,6 +2,8 @@ import { existsSync } from "fs";
|
|||||||
import grabDirNames from "../../utils/grab-dir-names";
|
import grabDirNames from "../../utils/grab-dir-names";
|
||||||
import grabConstants from "../../utils/grab-constants";
|
import grabConstants from "../../utils/grab-constants";
|
||||||
import grabRootFilePath from "../server/web-pages/grab-root-file-path";
|
import grabRootFilePath from "../server/web-pages/grab-root-file-path";
|
||||||
|
import grabRelativePath from "../../utils/grab-relative-path";
|
||||||
|
import { log } from "../../utils/log";
|
||||||
|
|
||||||
const { ROOT_DIR } = grabDirNames();
|
const { ROOT_DIR } = grabDirNames();
|
||||||
|
|
||||||
@@ -36,14 +38,25 @@ export default async function grabClientHydrationScript({
|
|||||||
|
|
||||||
const root_content = await Bun.file(root_file_path).text();
|
const root_content = await Bun.file(root_file_path).text();
|
||||||
|
|
||||||
if (!root_content.match(/^export default/m)) {
|
if (
|
||||||
|
!hasBundlableDefaultExport({
|
||||||
|
file_path: root_file_path,
|
||||||
|
content: root_content,
|
||||||
|
is_root: true,
|
||||||
|
})
|
||||||
|
) {
|
||||||
return undefined;
|
return undefined;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const page_content = await Bun.file(page_local_path).text();
|
const page_content = await Bun.file(page_local_path).text();
|
||||||
|
|
||||||
if (!page_content.match(/^export default/m)) {
|
if (
|
||||||
|
!hasBundlableDefaultExport({
|
||||||
|
file_path: page_local_path,
|
||||||
|
content: page_content,
|
||||||
|
})
|
||||||
|
) {
|
||||||
return undefined;
|
return undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -80,3 +93,54 @@ export default async function grabClientHydrationScript({
|
|||||||
|
|
||||||
return txt;
|
return txt;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// File path -> content hash of the last warning, so rebuilds don't repeat
|
||||||
|
// a warning until the file changes.
|
||||||
|
const warned_files = new Map<string, number | bigint>();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* # Check a page or root file can be bundled
|
||||||
|
* Files are only bundled when `export default` starts a line. Files with
|
||||||
|
* no default export at all (e.g. a page still being written) are skipped
|
||||||
|
* silently; this warns about the confusing case where one exists
|
||||||
|
* elsewhere, like `import X from "x"; export default X` on one line, or
|
||||||
|
* `export { Page as default }`.
|
||||||
|
*/
|
||||||
|
function hasBundlableDefaultExport({
|
||||||
|
file_path,
|
||||||
|
content,
|
||||||
|
is_root,
|
||||||
|
}: {
|
||||||
|
file_path: string;
|
||||||
|
content: string;
|
||||||
|
is_root?: boolean;
|
||||||
|
}): boolean {
|
||||||
|
if (/^export default/m.test(content)) {
|
||||||
|
warned_files.delete(file_path);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
const has_default_export =
|
||||||
|
/\bexport\s+default\b/.test(content) ||
|
||||||
|
/\bas\s+default\b/.test(content);
|
||||||
|
|
||||||
|
if (!has_default_export) {
|
||||||
|
warned_files.delete(file_path);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
const content_hash = Bun.hash(content);
|
||||||
|
if (warned_files.get(file_path) === content_hash) return false;
|
||||||
|
warned_files.set(file_path, content_hash);
|
||||||
|
|
||||||
|
log.warn(
|
||||||
|
`${grabRelativePath(file_path)} was skipped: ` +
|
||||||
|
`\`export default\` must be at the start of a line. ` +
|
||||||
|
(is_root
|
||||||
|
? `No pages can be bundled until the root layout is fixed.`
|
||||||
|
: `This page will not be bundled.`),
|
||||||
|
);
|
||||||
|
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
|||||||
+26
-2
@@ -523,10 +523,21 @@ export type BunextGenerateSitemapParams = {
|
|||||||
| BunextSitemapAdditionalPaths
|
| BunextSitemapAdditionalPaths
|
||||||
| Promise<BunextSitemapAdditionalPaths>);
|
| Promise<BunextSitemapAdditionalPaths>);
|
||||||
/**
|
/**
|
||||||
* Use each page file's modification time as `lastmod`. Defaults to
|
* Fall back to each page file's modification time as `lastmod` when
|
||||||
* `true`.
|
* the build record has no date for it. Defaults to `false`: checkouts
|
||||||
|
* reset modification times, so they are rarely meaningful.
|
||||||
*/
|
*/
|
||||||
use_file_lastmod?: boolean;
|
use_file_lastmod?: boolean;
|
||||||
|
/**
|
||||||
|
* `lastmod` for entries that have no other date
|
||||||
|
*/
|
||||||
|
default_lastmod?: Date | string;
|
||||||
|
/**
|
||||||
|
* Page last-modified record written by `bunext build`, relative to the
|
||||||
|
* project root. Defaults to `.bunext/page-lastmod.json`. Point it at a
|
||||||
|
* cached or committed location when builds start from a fresh checkout.
|
||||||
|
*/
|
||||||
|
lastmod_file?: string;
|
||||||
/**
|
/**
|
||||||
* Append a trailing slash to every non-root path. Defaults to `false`.
|
* Append a trailing slash to every non-root path. Defaults to `false`.
|
||||||
*/
|
*/
|
||||||
@@ -597,3 +608,16 @@ export type BunextRobotsConfig = BunextGenerateRobotsTxtParams & {
|
|||||||
*/
|
*/
|
||||||
output?: string;
|
output?: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Written by `bunext build`. Keyed by page path relative to the project
|
||||||
|
* root. `lastmod` is the build date at which `fingerprint` last changed.
|
||||||
|
*/
|
||||||
|
export type BunextPageLastmodRecord = {
|
||||||
|
pages: {
|
||||||
|
[local_path: string]: {
|
||||||
|
fingerprint: string;
|
||||||
|
lastmod: string;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import grabOrigin from "./grab-origin";
|
|||||||
import grabRelativePath from "./grab-relative-path";
|
import grabRelativePath from "./grab-relative-path";
|
||||||
import isDevelopment from "./is-development";
|
import isDevelopment from "./is-development";
|
||||||
import { log } from "./log";
|
import { log } from "./log";
|
||||||
|
import { readPageLastmodRecord } from "./page-lastmod-record";
|
||||||
|
|
||||||
const ErrorPagePaths = ["/404", "/500"];
|
const ErrorPagePaths = ["/404", "/500"];
|
||||||
|
|
||||||
@@ -201,7 +202,8 @@ function parseLastmod(lastmod?: Date | string) {
|
|||||||
export async function grabSitemapEntries(
|
export async function grabSitemapEntries(
|
||||||
params?: BunextGenerateSitemapParams,
|
params?: BunextGenerateSitemapParams,
|
||||||
): Promise<BunextSitemapEntry[]> {
|
): Promise<BunextSitemapEntry[]> {
|
||||||
const use_file_lastmod = params?.use_file_lastmod ?? true;
|
const use_file_lastmod = params?.use_file_lastmod ?? false;
|
||||||
|
const lastmod_record = readPageLastmodRecord(params?.lastmod_file);
|
||||||
const entries: BunextSitemapEntry[] = [];
|
const entries: BunextSitemapEntry[] = [];
|
||||||
|
|
||||||
const pages = grabAllPages({ exclude_api: true });
|
const pages = grabAllPages({ exclude_api: true });
|
||||||
@@ -218,8 +220,18 @@ export async function grabSitemapEntries(
|
|||||||
route: page_path,
|
route: page_path,
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// When the page's client bundle last changed, per `bunext build`
|
||||||
|
const build_lastmod =
|
||||||
|
lastmod_record?.pages[grabRelativePath(page.local_path)]?.lastmod;
|
||||||
|
|
||||||
if (page_sitemap_paths) {
|
if (page_sitemap_paths) {
|
||||||
entries.push(...page_sitemap_paths.map(toSitemapEntry));
|
for (const page_sitemap_path of page_sitemap_paths) {
|
||||||
|
const entry = toSitemapEntry(page_sitemap_path);
|
||||||
|
entries.push({
|
||||||
|
...entry,
|
||||||
|
lastmod: entry.lastmod || build_lastmod,
|
||||||
|
});
|
||||||
|
}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -227,9 +239,11 @@ export async function grabSitemapEntries(
|
|||||||
|
|
||||||
entries.push({
|
entries.push({
|
||||||
path: page_path,
|
path: page_path,
|
||||||
lastmod: use_file_lastmod
|
lastmod:
|
||||||
|
build_lastmod ||
|
||||||
|
(use_file_lastmod
|
||||||
? statSync(page.local_path).mtime
|
? statSync(page.local_path).mtime
|
||||||
: undefined,
|
: undefined),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -250,10 +264,11 @@ export async function grabSitemapEntries(
|
|||||||
seen.add(path);
|
seen.add(path);
|
||||||
|
|
||||||
final_entries.push({
|
final_entries.push({
|
||||||
changefreq: params?.default_changefreq,
|
|
||||||
priority: params?.default_priority,
|
|
||||||
...entry,
|
...entry,
|
||||||
path,
|
path,
|
||||||
|
lastmod: entry.lastmod || params?.default_lastmod,
|
||||||
|
changefreq: entry.changefreq || params?.default_changefreq,
|
||||||
|
priority: entry.priority ?? params?.default_priority,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ export type DirNames = {
|
|||||||
BUNDLER_CTX_MAP_JSON: string;
|
BUNDLER_CTX_MAP_JSON: string;
|
||||||
SSR_BUNDLER_CTX_MAP_JSON: string;
|
SSR_BUNDLER_CTX_MAP_JSON: string;
|
||||||
BUILD_INFO_JSON: string;
|
BUILD_INFO_JSON: string;
|
||||||
|
PAGE_LASTMOD_JSON: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
export default function grabDirNames(): DirNames {
|
export default function grabDirNames(): DirNames {
|
||||||
@@ -99,6 +100,7 @@ export default function grabDirNames(): DirNames {
|
|||||||
);
|
);
|
||||||
|
|
||||||
const BUILD_INFO_JSON = path.join(BUNX_CWD_DIR, "build-info.json");
|
const BUILD_INFO_JSON = path.join(BUNX_CWD_DIR, "build-info.json");
|
||||||
|
const PAGE_LASTMOD_JSON = path.join(BUNX_CWD_DIR, "page-lastmod.json");
|
||||||
|
|
||||||
return {
|
return {
|
||||||
ROOT_DIR,
|
ROOT_DIR,
|
||||||
@@ -131,6 +133,7 @@ export default function grabDirNames(): DirNames {
|
|||||||
BUNDLER_CTX_MAP_JSON,
|
BUNDLER_CTX_MAP_JSON,
|
||||||
SSR_BUNDLER_CTX_MAP_JSON,
|
SSR_BUNDLER_CTX_MAP_JSON,
|
||||||
BUILD_INFO_JSON,
|
BUILD_INFO_JSON,
|
||||||
|
PAGE_LASTMOD_JSON,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,70 @@
|
|||||||
|
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
|
||||||
|
import path from "path";
|
||||||
|
import type { BundlerCTXMap, BunextPageLastmodRecord } from "../types";
|
||||||
|
import grabDirNames from "./grab-dir-names";
|
||||||
|
import { log } from "./log";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* # Path of the page last-modified record
|
||||||
|
*/
|
||||||
|
export function grabPageLastmodFile(lastmod_file?: string) {
|
||||||
|
const { ROOT_DIR, PAGE_LASTMOD_JSON } = grabDirNames();
|
||||||
|
const target = lastmod_file || global.BUNEXT_CONFIG?.sitemap?.lastmod_file;
|
||||||
|
|
||||||
|
return target ? path.resolve(ROOT_DIR, target) : PAGE_LASTMOD_JSON;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function readPageLastmodRecord(
|
||||||
|
lastmod_file?: string,
|
||||||
|
): BunextPageLastmodRecord | undefined {
|
||||||
|
const file = grabPageLastmodFile(lastmod_file);
|
||||||
|
if (!existsSync(file)) return;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const record = JSON.parse(readFileSync(file, "utf-8"));
|
||||||
|
if (record?.pages && typeof record.pages == "object") return record;
|
||||||
|
} catch {}
|
||||||
|
|
||||||
|
log.warn(`Ignoring invalid page lastmod record ${file}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* # Record when each page's client bundle last changed
|
||||||
|
* A page's bundle hash covers its whole client import graph (including
|
||||||
|
* the root layout), so a component edit re-dates every page that uses it.
|
||||||
|
* Unchanged pages keep their previous date; removed pages are dropped.
|
||||||
|
*/
|
||||||
|
export function updatePageLastmodRecord({
|
||||||
|
ctx_map,
|
||||||
|
build_date = new Date(),
|
||||||
|
lastmod_file,
|
||||||
|
}: {
|
||||||
|
ctx_map: { [local_path: string]: BundlerCTXMap };
|
||||||
|
build_date?: Date;
|
||||||
|
lastmod_file?: string;
|
||||||
|
}): BunextPageLastmodRecord {
|
||||||
|
const prior = readPageLastmodRecord(lastmod_file);
|
||||||
|
const record: BunextPageLastmodRecord = { pages: {} };
|
||||||
|
|
||||||
|
// Sorted so a committed record produces minimal diffs
|
||||||
|
const local_paths = Object.keys(ctx_map).sort();
|
||||||
|
|
||||||
|
for (const local_path of local_paths) {
|
||||||
|
const artifact = ctx_map[local_path];
|
||||||
|
if (!artifact?.hash) continue;
|
||||||
|
|
||||||
|
const fingerprint = `${artifact.hash}:${artifact.css_path || ""}`;
|
||||||
|
const prior_page = prior?.pages[local_path];
|
||||||
|
|
||||||
|
record.pages[local_path] =
|
||||||
|
prior_page?.fingerprint == fingerprint
|
||||||
|
? prior_page
|
||||||
|
: { fingerprint, lastmod: build_date.toISOString() };
|
||||||
|
}
|
||||||
|
|
||||||
|
const file = grabPageLastmodFile(lastmod_file);
|
||||||
|
mkdirSync(path.dirname(file), { recursive: true });
|
||||||
|
writeFileSync(file, JSON.stringify(record, null, 4), "utf-8");
|
||||||
|
|
||||||
|
return record;
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user