Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gdocs): narrative charts component #4295

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
5fce282
feat: pre-fetch chart views metadata in gdocs
marcelgerber Dec 11, 2024
7ec093c
feat: NarrativeChart component
marcelgerber Dec 11, 2024
523c203
refactor: properly attach gdocs attachments
marcelgerber Dec 12, 2024
a6da14d
enhance: ability to filter `chartViewMetadata`
marcelgerber Dec 12, 2024
49a2c87
refactor: add narrative-chart to `extractGdocComponentInfo`
marcelgerber Dec 16, 2024
06a8e5a
refactor: chartViewMetadata -> narrativeViewInfo
marcelgerber Dec 17, 2024
4efe82c
enhance: narrative views are reflected as links in gdocs
marcelgerber Dec 17, 2024
41867ab
refactor: filter down `narrativeViewsInfo`
marcelgerber Dec 17, 2024
242bd2f
refactor: change `linkType` enum
marcelgerber Dec 17, 2024
c7ed3a1
fix: fix error when publishing NarrativeChart with error
marcelgerber Dec 18, 2024
b5b24a6
enhance: add narrative chart support to MultiEmbedder
marcelgerber Dec 18, 2024
cb37196
enhance: basic config to hide some grapher elements
marcelgerber Dec 18, 2024
afa899c
fix: correctly generate narrative view query params
marcelgerber Dec 18, 2024
5a834f8
refactor: use consistent names -- chart views & narrative charts
marcelgerber Dec 19, 2024
3cae254
feat: correctly generate query params for chart view
marcelgerber Dec 19, 2024
831da5b
style: remove unused import
marcelgerber Dec 19, 2024
8624a63
enhance: enable narrative charts on staging server
marcelgerber Dec 19, 2024
a6dc195
enhance: show static preview of chart view
marcelgerber Dec 19, 2024
10da0a2
style: remove unused import
marcelgerber Dec 19, 2024
be8aa57
enhance: make "explore the data" button blue
marcelgerber Dec 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion adminSiteClient/AdminSidebar.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ export const AdminSidebar = (): React.ReactElement => (
{chartViewsFeatureEnabled && (
<li>
<Link to="/chartViews">
<FontAwesomeIcon icon={faPanorama} /> Narrative views
<FontAwesomeIcon icon={faPanorama} /> Narrative charts
</Link>
</li>
)}
Expand Down
4 changes: 2 additions & 2 deletions adminSiteClient/ChartEditor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -200,15 +200,15 @@ export class ChartEditor extends AbstractChartEditor<ChartEditorManager> {
)
}

async saveAsNarrativeView(): Promise<void> {
async saveAsChartView(): Promise<void> {
const { patchConfig, grapher } = this

const chartJson = omit(patchConfig, CHART_VIEW_PROPS_TO_OMIT)

const suggestedName = grapher.title ? slugify(grapher.title) : undefined

const name = prompt(
"Please enter a programmatic name for the narrative view. Note that this name cannot be changed later.",
"Please enter a programmatic name for the narrative chart. Note that this name cannot be changed later.",
suggestedName
)

Expand Down
5 changes: 3 additions & 2 deletions adminSiteClient/ChartViewEditor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import {
References,
type EditorTab,
} from "./AbstractChartEditor.js"
import { ENV } from "../settings/clientSettings.js"
import { BAKED_BASE_URL, ENV } from "../settings/clientSettings.js"
import {
CHART_VIEW_PROPS_TO_OMIT,
CHART_VIEW_PROPS_TO_PERSIST,
Expand All @@ -16,7 +16,8 @@ import { diffGrapherConfigs, omit, pick } from "@ourworldindata/utils"
// Don't yet show chart views in the admin interface
// This is low-stakes - if it shows up anyhow (e.g. on staging servers), it's not a big deal.
// TODO: Remove this flag once we're launching this feature
export const chartViewsFeatureEnabled = ENV === "development"
export const chartViewsFeatureEnabled =
ENV === "development" || BAKED_BASE_URL.includes("narrative-")

export interface Chart {
id: number
Expand Down
6 changes: 3 additions & 3 deletions adminSiteClient/ChartViewIndexPage.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { AdminAppContext } from "./AdminAppContext.js"
import { Timeago } from "./Forms.js"
import { ColumnsType } from "antd/es/table/InternalTable.js"
import { ApiChartViewOverview } from "../adminShared/AdminTypes.js"
import { BAKED_GRAPHER_URL } from "../settings/clientSettings.js"
import { GRAPHER_DYNAMIC_THUMBNAIL_URL } from "../settings/clientSettings.js"
import { Link } from "./Link.js"
import {
buildSearchWordsFromSearchString,
Expand All @@ -27,7 +27,7 @@ function createColumns(
width: 200,
render: (chartConfigId) => (
<img
src={`${BAKED_GRAPHER_URL}/by-uuid/${chartConfigId}.svg`}
src={`${GRAPHER_DYNAMIC_THUMBNAIL_URL}/by-uuid/${chartConfigId}.svg`}
style={{ maxWidth: 200, maxHeight: 200 }}
/>
),
Expand Down Expand Up @@ -134,7 +134,7 @@ export function ChartViewIndexPage() {
}, [admin])

return (
<AdminLayout title="Narrative views">
<AdminLayout title="Narrative charts">
<main className="ChartViewIndexPage">
<Flex justify="space-between">
<Input
Expand Down
2 changes: 1 addition & 1 deletion adminSiteClient/EditorReferencesTab.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ export const ReferencesSection = (props: {

const chartViews = !!props.references?.chartViews?.length && (
<>
<p>Narrative views based on this chart</p>
<p>Narrative charts based on this chart</p>
<ul className="list-group">
{props.references.chartViews.map((chartView) => (
<li key={chartView.id} className="list-group-item">
Expand Down
8 changes: 4 additions & 4 deletions adminSiteClient/SaveButtons.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,8 @@ class SaveButtonsForChart extends React.Component<{
void this.props.editor.saveAsNewGrapher()
}

@action.bound onSaveAsNarrativeView() {
void this.props.editor.saveAsNarrativeView()
@action.bound onSaveAsChartView() {
void this.props.editor.saveAsChartView()
}

@action.bound onPublishToggle() {
Expand Down Expand Up @@ -122,10 +122,10 @@ class SaveButtonsForChart extends React.Component<{
<div className="mt-2">
<button
className="btn btn-primary"
onClick={this.onSaveAsNarrativeView}
onClick={this.onSaveAsChartView}
disabled={isSavingDisabled}
>
Save as narrative view
Save as narrative chart
</button>
</div>
)}
Expand Down
2 changes: 1 addition & 1 deletion adminSiteServer/apiRouter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3637,7 +3637,7 @@ const createPatchConfigAndQueryParamsForChartView = async (
...pick(fullConfigIncludingDefaults, CHART_VIEW_PROPS_TO_PERSIST),
}

const queryParams = grapherConfigToQueryParams(config)
const queryParams = grapherConfigToQueryParams(patchConfigToSave)

const fullConfig = mergeGrapherConfigs(parentChartConfig, patchConfigToSave)
return { patchConfig: patchConfigToSave, fullConfig, queryParams }
Expand Down
23 changes: 21 additions & 2 deletions baker/SiteBaker.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ import {
grabMetadataForGdocLinkedIndicator,
TombstonePageData,
gdocUrlRegex,
ChartViewInfo,
} from "@ourworldindata/utils"
import { execWrapper } from "../db/execWrapper.js"
import { countryProfileSpecs } from "../site/countryProfileProjects.js"
Expand Down Expand Up @@ -109,6 +110,7 @@ import { getTombstones } from "../db/model/GdocTombstone.js"
import { bakeAllMultiDimDataPages } from "./MultiDimBaker.js"
import { getAllLinkedPublishedMultiDimDataPages } from "../db/model/MultiDimDataPage.js"
import { getPublicDonorNames } from "../db/model/Donor.js"
import { getChartViewsInfo } from "../db/model/ChartView.js"

type PrefetchedAttachments = {
donors: string[]
Expand All @@ -120,6 +122,7 @@ type PrefetchedAttachments = {
explorers: Record<string, LinkedChart>
}
linkedIndicators: Record<number, LinkedIndicator>
linkedChartViews: Record<string, ChartViewInfo>
}

// These aren't all "wordpress" steps
Expand Down Expand Up @@ -176,7 +179,7 @@ function getProgressBarTotal(bakeSteps: BakeStepConfig): number {
bakeSteps.has("dataInsights") ||
bakeSteps.has("authors")
) {
total += 8
total += 9
}
return total
}
Expand Down Expand Up @@ -345,7 +348,7 @@ export class SiteBaker {
_prefetchedAttachmentsCache: PrefetchedAttachments | undefined = undefined
private async getPrefetchedGdocAttachments(
knex: db.KnexReadonlyTransaction,
picks?: [string[], string[], string[], string[], string[]]
picks?: [string[], string[], string[], string[], string[], string[]]
): Promise<PrefetchedAttachments> {
if (!this._prefetchedAttachmentsCache) {
console.log("Prefetching attachments...")
Expand Down Expand Up @@ -459,6 +462,12 @@ export class SiteBaker {
name: `✅ Prefetched ${publishedAuthors.length} authors`,
})

const chartViewsInfo = await getChartViewsInfo(knex)
const chartViewsInfoByName = keyBy(chartViewsInfo, "name")
this.progressBar.tick({
name: `✅ Prefetched ${chartViewsInfo.length} chart views`,
})

const prefetchedAttachments = {
donors,
linkedAuthors: publishedAuthors,
Expand All @@ -469,6 +478,7 @@ export class SiteBaker {
graphers: publishedChartsBySlug,
},
linkedIndicators: datapageIndicatorsById,
linkedChartViews: chartViewsInfoByName,
}
this.progressBar.tick({ name: "✅ Prefetched attachments" })
this._prefetchedAttachmentsCache = prefetchedAttachments
Expand All @@ -480,6 +490,7 @@ export class SiteBaker {
imageFilenames,
linkedGrapherSlugs,
linkedExplorerSlugs,
linkedChartViewNames,
] = picks
const linkedDocuments = pick(
this._prefetchedAttachmentsCache.linkedDocuments,
Expand Down Expand Up @@ -528,6 +539,10 @@ export class SiteBaker {
this._prefetchedAttachmentsCache.linkedAuthors.filter(
(author) => authorNames.includes(author.name)
),
linkedChartViews: pick(
this._prefetchedAttachmentsCache.linkedChartViews,
linkedChartViewNames
),
}
}
return this._prefetchedAttachmentsCache
Expand Down Expand Up @@ -619,6 +634,7 @@ export class SiteBaker {
publishedGdoc.linkedImageFilenames,
publishedGdoc.linkedChartSlugs.grapher,
publishedGdoc.linkedChartSlugs.explorer,
publishedGdoc.linkedChartViewNames,
])
publishedGdoc.donors = attachments.donors
publishedGdoc.linkedAuthors = attachments.linkedAuthors
Expand All @@ -629,6 +645,7 @@ export class SiteBaker {
...attachments.linkedCharts.explorers,
}
publishedGdoc.linkedIndicators = attachments.linkedIndicators
publishedGdoc.linkedChartViews = attachments.linkedChartViews

// this is a no-op if the gdoc doesn't have an all-chart block
if ("loadRelatedCharts" in publishedGdoc) {
Expand Down Expand Up @@ -876,6 +893,7 @@ export class SiteBaker {
dataInsight.linkedImageFilenames,
dataInsight.linkedChartSlugs.grapher,
dataInsight.linkedChartSlugs.explorer,
dataInsight.linkedChartViewNames,
])
dataInsight.linkedDocuments = attachments.linkedDocuments
dataInsight.imageMetadata = {
Expand Down Expand Up @@ -949,6 +967,7 @@ export class SiteBaker {
publishedAuthor.linkedImageFilenames,
publishedAuthor.linkedChartSlugs.grapher,
publishedAuthor.linkedChartSlugs.explorer,
publishedAuthor.linkedChartViewNames,
])

// We don't need these to be attached to the gdoc in the current
Expand Down
1 change: 1 addition & 0 deletions baker/siteRenderers.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -439,6 +439,7 @@ ${dataInsights
latestDataInsights: get(post, "latestDataInsights", []),
homepageMetadata: get(post, "homepageMetadata", {}),
latestWorkLinks: get(post, "latestWorkLinks", []),
linkedChartViews: get(post, "linkedChartViews", {}),
}}
>
<AtomArticleBlocks blocks={post.content.body} />
Expand Down
17 changes: 17 additions & 0 deletions db/migration/1734454799588-PostsGdocsLinksAddChartViews.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { MigrationInterface, QueryRunner } from "typeorm"

export class PostsGdocsLinksAddChartViews1734454799588
implements MigrationInterface
{
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`
ALTER TABLE posts_gdocs_links
MODIFY linkType ENUM ('gdoc', 'url', 'grapher', 'explorer', 'chart-view') NULL`)
}

public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`
ALTER TABLE posts_gdocs_links
MODIFY linkType ENUM ('gdoc', 'url', 'grapher', 'explorer') NULL`)
}
}
34 changes: 34 additions & 0 deletions db/model/ChartView.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { ChartViewInfo, JsonString } from "@ourworldindata/types"
import * as db from "../db.js"

export const getChartViewsInfo = async (
knex: db.KnexReadonlyTransaction,
names?: string[]
): Promise<ChartViewInfo[]> => {
type RawRow = Omit<ChartViewInfo, "queryParamsForParentChart"> & {
queryParamsForParentChart: JsonString
}
let rows: RawRow[]

const query = `-- sql
SELECT cv.name,
cc.full ->> "$.title" as title,
chartConfigId,
pcc.slug as parentChartSlug,
cv.queryParamsForParentChart
FROM chart_views cv
JOIN chart_configs cc on cc.id = cv.chartConfigId
JOIN charts pc on cv.parentChartId = pc.id
JOIN chart_configs pcc on pc.configId = pcc.id
`

if (names) {
if (names.length === 0) return []
rows = await db.knexRaw(knex, `${query} WHERE cv.name IN (?)`, [names])
} else rows = await db.knexRaw(knex, query)

return rows.map((row) => ({
...row,
queryParamsForParentChart: JSON.parse(row.queryParamsForParentChart),
}))
}
26 changes: 25 additions & 1 deletion db/model/Gdoc/GdocBase.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,15 @@ import {
getVariableMetadata,
getVariableOfDatapageIfApplicable,
} from "../Variable.js"
import { createLinkFromUrl } from "../Link.js"
import { createLinkForChartView, createLinkFromUrl } from "../Link.js"
import {
getMultiDimDataPageBySlug,
isMultiDimDataPagePublished,
} from "../MultiDimDataPage.js"
import {
ARCHVED_THUMBNAIL_FILENAME,
ChartConfigType,
ChartViewInfo,
DEFAULT_THUMBNAIL_FILENAME,
GrapherInterface,
LatestDataInsight,
Expand All @@ -66,6 +67,7 @@ import {
OwidGdocLinkType,
OwidGdocType,
} from "@ourworldindata/types"
import { getChartViewsInfo } from "../ChartView.js"

export class GdocBase implements OwidGdocBaseInterface {
id!: string
Expand All @@ -89,6 +91,7 @@ export class GdocBase implements OwidGdocBaseInterface {
linkedIndicators: Record<number, LinkedIndicator> = {}
linkedDocuments: Record<string, OwidGdocMinimalPostInterface> = {}
latestDataInsights: LatestDataInsight[] = []
linkedChartViews?: Record<string, ChartViewInfo> = {}
_omittableFields: string[] = []

constructor(id?: string) {
Expand Down Expand Up @@ -292,6 +295,14 @@ export class GdocBase implements OwidGdocBaseInterface {
return { grapher: [...grapher], explorer: [...explorer] }
}

get linkedChartViewNames(): string[] {
const filteredLinks = this.links
.filter((link) => link.linkType === OwidGdocLinkType.ChartView)
.map((link) => link.target)

return filteredLinks
}

get hasAllChartsBlock(): boolean {
let hasAllChartsBlock = false
for (const enrichedBlockSource of this.enrichedBlockSources) {
Expand Down Expand Up @@ -338,6 +349,13 @@ export class GdocBase implements OwidGdocBaseInterface {
componentType: block.type,
}),
])
.with({ type: "narrative-chart" }, (block) => [
createLinkForChartView({
name: block.name,
source: this,
componentType: block.type,
}),
])
.with({ type: "all-charts" }, (block) =>
block.top.map((item) =>
createLinkFromUrl({
Expand Down Expand Up @@ -700,6 +718,11 @@ export class GdocBase implements OwidGdocBaseInterface {
}
}

async loadChartViewsInfo(knex: db.KnexReadonlyTransaction): Promise<void> {
const result = await getChartViewsInfo(knex, this.linkedChartViewNames)
this.linkedChartViews = keyBy(result, "name")
}

async fetchAndEnrichGdoc(): Promise<void> {
const docsClient = google.docs({
version: "v1",
Expand Down Expand Up @@ -845,6 +868,7 @@ export class GdocBase implements OwidGdocBaseInterface {
await this.loadImageMetadataFromDB(knex)
await this.loadLinkedCharts(knex)
await this.loadLinkedIndicators() // depends on linked charts
await this.loadChartViewsInfo(knex)
await this._loadSubclassAttachments(knex)
await this.validate(knex)
}
Expand Down
11 changes: 11 additions & 0 deletions db/model/Gdoc/enrichedToMarkdown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,17 @@ ${items}
exportComponents
)
)
.with({ type: "narrative-chart" }, (b): string | undefined =>
markdownComponent(
"NarrativeChart",
{
name: b.name,
caption: b.caption ? spansToMarkdown(b.caption) : undefined,
// Note: truncated
},
exportComponents
)
)
.with({ type: "code" }, (b): string | undefined => {
return (
"```\n" +
Expand Down
Loading
Loading