Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add robots.txt and sitemap.xml #456

Merged
merged 4 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions app/routes/_seo+/robots[.]txt.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { generateRobotsTxt } from '@nasa-gcn/remix-seo'
import { type DataFunctionArgs } from '@remix-run/node'
import { getDomainUrl } from '#app/utils/misc.tsx'

export function loader({ request }: DataFunctionArgs) {
return generateRobotsTxt([
{ type: 'sitemap', value: `${getDomainUrl(request)}/sitemap.xml` },
])
}
14 changes: 14 additions & 0 deletions app/routes/_seo+/sitemap[.]xml.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { generateSitemap } from '@nasa-gcn/remix-seo'
// @ts-expect-error - this does work, though it's not exactly a public API
import { routes } from '@remix-run/dev/server-build'
import { type DataFunctionArgs } from '@remix-run/node'
import { getDomainUrl } from '#app/utils/misc.tsx'

export function loader({ request }: DataFunctionArgs) {
return generateSitemap(request, routes, {
siteUrl: getDomainUrl(request),
headers: {
'Cache-Control': `public, max-age=${60 * 5}`,
},
})
}
5 changes: 5 additions & 0 deletions app/routes/admin+/cache.tsx
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { json, redirect, type DataFunctionArgs } from '@remix-run/node'
import {
Form,
Expand Down Expand Up @@ -28,6 +29,10 @@ import {
} from '#app/utils/misc.tsx'
import { requireUserWithRole } from '#app/utils/permissions.ts'

export const handle: SEOHandle = {
getSitemapEntries: () => null,
}

export async function loader({ request }: DataFunctionArgs) {
await requireUserWithRole(request, 'admin')
const searchParams = new URL(request.url).searchParams
Expand Down
5 changes: 4 additions & 1 deletion app/routes/settings+/profile.change-email.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { conform, useForm } from '@conform-to/react'
import { getFieldsetConstraint, parse } from '@conform-to/zod'
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import * as E from '@react-email/components'
import { json, redirect, type DataFunctionArgs } from '@remix-run/node'
import { Form, useActionData, useLoaderData } from '@remix-run/react'
Expand All @@ -19,9 +20,11 @@ import { invariant, useIsPending } from '#app/utils/misc.tsx'
import { redirectWithToast } from '#app/utils/toast.server.ts'
import { EmailSchema } from '#app/utils/user-validation.ts'
import { verifySessionStorage } from '#app/utils/verification.server.ts'
import { type BreadcrumbHandle } from './profile.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="envelope-closed">Change Email</Icon>,
getSitemapEntries: () => null,
}

const newEmailAddressSessionKey = 'new-email-address'
Expand Down
7 changes: 5 additions & 2 deletions app/routes/settings+/profile.connections.tsx
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import {
json,
type DataFunctionArgs,
Expand All @@ -17,17 +18,19 @@ import { requireUserId } from '#app/utils/auth.server.ts'
import { resolveConnectionData } from '#app/utils/connections.server.ts'
import {
ProviderConnectionForm,
ProviderName,
type ProviderName,
ProviderNameSchema,
providerIcons,
providerNames,
} from '#app/utils/connections.tsx'
import { prisma } from '#app/utils/db.server.ts'
import { invariantResponse } from '#app/utils/misc.tsx'
import { createToastHeaders } from '#app/utils/toast.server.ts'
import { type BreadcrumbHandle } from './profile.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="link-2">Connections</Icon>,
getSitemapEntries: () => null,
}

async function userCanDeleteConnections(userId: string) {
Expand Down
5 changes: 5 additions & 0 deletions app/routes/settings+/profile.index.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { conform, useForm } from '@conform-to/react'
import { getFieldsetConstraint, parse } from '@conform-to/zod'
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { json, redirect, type DataFunctionArgs } from '@remix-run/node'
import { Link, useFetcher, useLoaderData } from '@remix-run/react'
import { z } from 'zod'
Expand All @@ -18,6 +19,10 @@ import { sessionStorage } from '#app/utils/session.server.ts'
import { NameSchema, UsernameSchema } from '#app/utils/user-validation.ts'
import { twoFAVerificationType } from './profile.two-factor.tsx'

export const handle: SEOHandle = {
getSitemapEntries: () => null,
}

const ProfileFormSchema = z.object({
name: NameSchema.optional(),
username: UsernameSchema,
Expand Down
5 changes: 4 additions & 1 deletion app/routes/settings+/profile.password.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { conform, useForm } from '@conform-to/react'
import { getFieldsetConstraint, parse } from '@conform-to/zod'
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { json, redirect, type DataFunctionArgs } from '@remix-run/node'
import { Form, Link, useActionData } from '@remix-run/react'
import { z } from 'zod'
Expand All @@ -16,9 +17,11 @@ import { prisma } from '#app/utils/db.server.ts'
import { useIsPending } from '#app/utils/misc.tsx'
import { redirectWithToast } from '#app/utils/toast.server.ts'
import { PasswordSchema } from '#app/utils/user-validation.ts'
import { type BreadcrumbHandle } from './profile.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="dots-horizontal">Password</Icon>,
getSitemapEntries: () => null,
}

const ChangePasswordForm = z
Expand Down
5 changes: 4 additions & 1 deletion app/routes/settings+/profile.password_.create.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { conform, useForm } from '@conform-to/react'
import { getFieldsetConstraint, parse } from '@conform-to/zod'
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { json, redirect, type DataFunctionArgs } from '@remix-run/node'
import { Form, Link, useActionData } from '@remix-run/react'
import { z } from 'zod'
Expand All @@ -11,9 +12,11 @@ import { getPasswordHash, requireUserId } from '#app/utils/auth.server.ts'
import { prisma } from '#app/utils/db.server.ts'
import { useIsPending } from '#app/utils/misc.tsx'
import { PasswordSchema } from '#app/utils/user-validation.ts'
import { type BreadcrumbHandle } from './profile.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="dots-horizontal">Password</Icon>,
getSitemapEntries: () => null,
}

const CreatePasswordForm = z
Expand Down
5 changes: 4 additions & 1 deletion app/routes/settings+/profile.photo.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { conform, useForm } from '@conform-to/react'
import { getFieldsetConstraint, parse } from '@conform-to/zod'
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import {
json,
redirect,
Expand All @@ -23,9 +24,11 @@ import {
useDoubleCheck,
useIsPending,
} from '#app/utils/misc.tsx'
import { type BreadcrumbHandle } from './profile.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="avatar">Photo</Icon>,
getSitemapEntries: () => null,
}

const MAX_SIZE = 1024 * 1024 * 3 // 3MB
Expand Down
9 changes: 7 additions & 2 deletions app/routes/settings+/profile.tsx
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { json, type DataFunctionArgs } from '@remix-run/node'
import { Link, Outlet, useMatches } from '@remix-run/react'
import { z } from 'zod'
Expand All @@ -8,8 +9,12 @@ import { prisma } from '#app/utils/db.server.ts'
import { cn, invariantResponse } from '#app/utils/misc.tsx'
import { useUser } from '#app/utils/user.ts'

export const handle = {
export const BreadcrumbHandle = z.object({ breadcrumb: z.any() })
export type BreadcrumbHandle = z.infer<typeof BreadcrumbHandle>

export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="file-text">Edit Profile</Icon>,
getSitemapEntries: () => null,
}

export async function loader({ request }: DataFunctionArgs) {
Expand All @@ -23,7 +28,7 @@ export async function loader({ request }: DataFunctionArgs) {
}

const BreadcrumbHandleMatch = z.object({
handle: z.object({ breadcrumb: z.any() }),
handle: BreadcrumbHandle,
})

export default function EditUserProfile() {
Expand Down
5 changes: 4 additions & 1 deletion app/routes/settings+/profile.two-factor.disable.tsx
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { json, type DataFunctionArgs } from '@remix-run/node'
import { useFetcher } from '@remix-run/react'
import { Icon } from '#app/components/ui/icon.tsx'
Expand All @@ -7,10 +8,12 @@ import { requireUserId } from '#app/utils/auth.server.ts'
import { prisma } from '#app/utils/db.server.ts'
import { useDoubleCheck } from '#app/utils/misc.tsx'
import { redirectWithToast } from '#app/utils/toast.server.ts'
import { type BreadcrumbHandle } from './profile.tsx'
import { twoFAVerificationType } from './profile.two-factor.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="lock-open-1">Disable</Icon>,
getSitemapEntries: () => null,
}

export async function loader({ request }: DataFunctionArgs) {
Expand Down
5 changes: 5 additions & 0 deletions app/routes/settings+/profile.two-factor.index.tsx
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { generateTOTP } from '@epic-web/totp'
import { SEOHandle } from '@nasa-gcn/remix-seo'

Check warning on line 2 in app/routes/settings+/profile.two-factor.index.tsx

View workflow job for this annotation

GitHub Actions / ⬣ ESLint

All imports in the declaration are only used as types. Use `import type`
import { json, redirect, type DataFunctionArgs } from '@remix-run/node'
import { Link, useFetcher, useLoaderData } from '@remix-run/react'
import { Icon } from '#app/components/ui/icon.tsx'
Expand All @@ -8,6 +9,10 @@
import { twoFAVerificationType } from './profile.two-factor.tsx'
import { twoFAVerifyVerificationType } from './profile.two-factor.verify.tsx'

export const handle: SEOHandle = {
getSitemapEntries: () => null,
}

export async function loader({ request }: DataFunctionArgs) {
const userId = await requireUserId(request)
const verification = await prisma.verification.findUnique({
Expand Down
5 changes: 4 additions & 1 deletion app/routes/settings+/profile.two-factor.tsx
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { Outlet } from '@remix-run/react'
import { Icon } from '#app/components/ui/icon.tsx'
import { type VerificationTypes } from '#app/routes/_auth+/verify.tsx'
import { type BreadcrumbHandle } from './profile.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="lock-closed">2FA</Icon>,
getSitemapEntries: () => null,
}

export const twoFAVerificationType = '2fa' satisfies VerificationTypes
Expand Down
5 changes: 4 additions & 1 deletion app/routes/settings+/profile.two-factor.verify.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { conform, useForm } from '@conform-to/react'
import { getFieldsetConstraint, parse } from '@conform-to/zod'
import { getTOTPAuthUri } from '@epic-web/totp'
import { type SEOHandle } from '@nasa-gcn/remix-seo'
import { json, redirect, type DataFunctionArgs } from '@remix-run/node'
import {
Form,
Expand All @@ -18,10 +19,12 @@ import { requireUserId } from '#app/utils/auth.server.ts'
import { prisma } from '#app/utils/db.server.ts'
import { getDomainUrl, useIsPending } from '#app/utils/misc.tsx'
import { redirectWithToast } from '#app/utils/toast.server.ts'
import { type BreadcrumbHandle } from './profile.tsx'
import { twoFAVerificationType } from './profile.two-factor.tsx'

export const handle = {
export const handle: BreadcrumbHandle & SEOHandle = {
breadcrumb: <Icon name="check">Verify</Icon>,
getSitemapEntries: () => null,
}

const VerifySchema = z.object({
Expand Down
4 changes: 0 additions & 4 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@ This page links to examples of how to implement some things with the Epic Stack.
by [@kentcdodds](https://github.com/kentcdodds): Using client hints to avoid
content layout shift with `prefers-reduced-motion` and framer motion
animations.
- [Sitemaps](https://github.com/kentcdodds/epic-stack-with-sitemap) by
[@kentcdodds](https://github.com/kentcdodds): Automatically generating a
sitemap and a nice way to handle dynamic routes and customize the sitemap on a
per-route basis.
- [Cross-site Request Forgery Protection (CSRF)](https://github.com/kentcdodds/epic-stack-with-csrf)
by [@kentcdodds](https://github.com/kentcdodds): An example of the Epic Stack
with CSRF protection on forms.
Expand Down
41 changes: 41 additions & 0 deletions docs/seo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# SEO

Remix has built-in support for setting up `meta` tags on a per-route basis which
you can read about
[in the Remix Metadata docs](https://remix.run/docs/en/main/route/meta).

The Epic Stack also has built-in support for `/robots.txt` and `/sitemap.xml`
via [resource routes](https://remix.run/docs/en/main/guides/resource-routes)
using [`@nasa-gcn/remix-seo`](https://github.com/nasa-gcn/remix-seo). By
default, all routes are included in the `sitemap.xml` file, but you can
configure which routes are included using the `handle` export in the route. Only
public-facing pages should be included in the `sitemap.xml` file.

Here are two quick examples of how to customize the sitemap on a per-route basis
from the `@nasa-gcn/remix-seo` docs:

```tsx
// routes/blog/$blogslug.tsx

export const handle: SEOHandle = {
getSitemapEntries: async request => {
const blogs = await db.blog.findMany()
return blogs.map(blog => {
return { route: `/blog/${blog.slug}`, priority: 0.7 }
})
},
}
```

```tsx
// in your routes/url-that-doesnt-need-sitemap
import { SEOHandle } from '@nasa-gcn/remix-seo'

export let loader: LoaderFunction = ({ request }) => {
/**/
}

export const handle: SEOHandle = {
getSitemapEntries: () => null,
}
```
16 changes: 14 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
"@conform-to/zod": "^0.9.0",
"@epic-web/remember": "^1.0.2",
"@epic-web/totp": "^1.1.1",
"@nasa-gcn/remix-seo": "^2.0.0",
"@paralleldrive/cuid2": "^2.2.2",
"@prisma/client": "^5.3.1",
"@radix-ui/react-checkbox": "^1.0.4",
Expand Down
2 changes: 0 additions & 2 deletions public/robots.txt

This file was deleted.