Skip to content

Commit

Permalink
Merge pull request #733 from microlinkhq/x
Browse files Browse the repository at this point in the history 
test(x): update snapshots
  • Loading branch information
@Kikobeats
Kikobeats authored Dec 7, 2024
2 parents 814e8e2 + c801564 commit d126e2e
Show file tree
Hide file tree
Showing 11 changed files with 117 additions and 401 deletions.
2 changes: 2 additions & 0 deletions packages/metascraper-x/test/fixtures/post-gif.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
<!DOCTYPE html><html dir="ltr" lang="en"><head><meta charset="utf-8" /><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0,viewport-fit=cover" /><link rel="preconnect" href="//abs.twimg.com" /><link rel="dns-prefetch" href="//abs.twimg.com" /><link rel="preconnect" href="//api.twitter.com" /><link rel="dns-prefetch" href="//api.twitter.com" /><link rel="preconnect" href="//api.x.com" /><link rel="dns-prefetch" href="//api.x.com" /><link rel="preconnect" href="//pbs.twimg.com" /><link rel="dns-prefetch" href="//pbs.twimg.com" /><link rel="preconnect" href="//t.co" /><link rel="dns-prefetch" href="//t.co" /><link rel="preconnect" href="//video.twimg.com" /><link rel="dns-prefetch" href="//video.twimg.com" /><meta http-equiv="onion-location" content="https://twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion/" /><meta property="fb:app_id" content="2231777543" /><meta content="X (formerly Twitter)" property="og:site_name" /><meta name="google-site-verification" content="600dQ0pZYsH2xOFt4hYmf5f5NpjCbWE_qk5Y04dErYM" /><meta name="facebook-domain-verification" content="x6sdcc8b5ju3bh8nbm59eswogvg6t1" /><meta name="mobile-web-app-capable" content="yes" /><meta name="apple-mobile-web-app-title" content="Twitter" /><meta name="apple-mobile-web-app-status-bar-style" content="white" /><link rel="search" type="application/opensearchdescription+xml" href="/os-x.xml" title="X">
<link rel="search" type="application/opensearchdescription+xml" href="/os-grok.xml" title="Grok"><link rel="manifest" href="/manifest.json" crossOrigin="use-credentials" /><link rel="search" type="application/opensearchdescription+xml" href="/os-x.xml" title="X"><link rel="search" type="application/opensearchdescription+xml" href="/os-grok.xml" title="Grok"><link rel="shortcut icon" href="//abs.twimg.com/favicons/twitter.3.ico"><meta name="theme-color" media="(prefers-color-scheme: light)" content="#FFFFFF" /><meta name="theme-color" media="(prefers-color-scheme: dark)" content="#000000" /><script type="application/ld+json">{"@context":"https://schema.org","@type":"WebSite","url":"https://twitter.com/","potentialAction":{"@type":"SearchAction","query-input":"required name=search_term_string","target":{"@type":"EntryPoint","urlTemplate":"https://twitter.com/search?q={search_term_string}&ref_src=twcamp%5Eseo_searchbox%7Ctwsrc%5Eseo"}}}</script><meta http-equiv="origin-trial" content="AlpCmb40F5ZjDi9ZYe+wnr/V8MF+XmY41K4qUhoq+2mbepJTNd3q4CRqlACfnythEPZqcjryfAS1+ExS0FFRcA8AAABmeyJvcmlnaW4iOiJodHRwczovL3R3aXR0ZXIuY29tOjQ0MyIsImZlYXR1cmUiOiJMYXVuY2ggSGFuZGxlciIsImV4cGlyeSI6MTY1NTI1MTE5OSwiaXNTdWJkb21haW4iOnRydWV9" /><style>html,body{height: 100%;}::cue{white-space:normal}</style><meta content="article" property="og:type" /><meta content="https://twitter.com/Kikobeats/status/880139124791029763" property="og:url" /><meta content="#!/kiko/beats (@Kikobeats) on X" property="og:title" /><meta content="Experimenting with Clearbit API + Apple TV 3D Parallax https://t.co/Qsm163k4mJ" property="og:description" /><meta content="https://pbs.twimg.com/tweet_video_thumb/DDbh3WCXYAAZfz9.jpg:large" property="og:image" /><meta content="Experimenting with Clearbit API + Apple TV 3D Parallax https://t.co/Qsm163k4mJ" name="description" /><meta content="#!/kiko/beats (@Kikobeats) on X" name="title" /><meta content="twitter://status?id=880139124791029763" property="al:ios:url" /><meta content="333903271" property="al:ios:app_store_id" /><meta content="X" property="al:ios:app_name" /><meta content="twitter://status?id=880139124791029763" property="al:android:url" /><meta content="com.twitter.android" property="al:android:package" /><meta content="X" property="al:android:app_name" /></head><body style="background-color: #FFFFFF;"><div id="react-root" style="height:100%;display:flex;"></div>
2 changes: 2 additions & 0 deletions packages/metascraper-x/test/fixtures/post-image.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
<!DOCTYPE html><html dir="ltr" lang="en"><head><meta charset="utf-8" /><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0,viewport-fit=cover" /><link rel="preconnect" href="//abs.twimg.com" /><link rel="dns-prefetch" href="//abs.twimg.com" /><link rel="preconnect" href="//api.twitter.com" /><link rel="dns-prefetch" href="//api.twitter.com" /><link rel="preconnect" href="//api.x.com" /><link rel="dns-prefetch" href="//api.x.com" /><link rel="preconnect" href="//pbs.twimg.com" /><link rel="dns-prefetch" href="//pbs.twimg.com" /><link rel="preconnect" href="//t.co" /><link rel="dns-prefetch" href="//t.co" /><link rel="preconnect" href="//video.twimg.com" /><link rel="dns-prefetch" href="//video.twimg.com" /><meta http-equiv="onion-location" content="https://twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion/" /><meta property="fb:app_id" content="2231777543" /><meta content="X (formerly Twitter)" property="og:site_name" /><meta name="google-site-verification" content="600dQ0pZYsH2xOFt4hYmf5f5NpjCbWE_qk5Y04dErYM" /><meta name="facebook-domain-verification" content="x6sdcc8b5ju3bh8nbm59eswogvg6t1" /><meta name="mobile-web-app-capable" content="yes" /><meta name="apple-mobile-web-app-title" content="Twitter" /><meta name="apple-mobile-web-app-status-bar-style" content="white" /><link rel="search" type="application/opensearchdescription+xml" href="/os-x.xml" title="X">
<link rel="search" type="application/opensearchdescription+xml" href="/os-grok.xml" title="Grok"><link rel="manifest" href="/manifest.json" crossOrigin="use-credentials" /><link rel="search" type="application/opensearchdescription+xml" href="/os-x.xml" title="X"><link rel="search" type="application/opensearchdescription+xml" href="/os-grok.xml" title="Grok"><link rel="shortcut icon" href="//abs.twimg.com/favicons/twitter.3.ico"><meta name="theme-color" media="(prefers-color-scheme: light)" content="#FFFFFF" /><meta name="theme-color" media="(prefers-color-scheme: dark)" content="#000000" /><script type="application/ld+json">{"@context":"https://schema.org","@type":"WebSite","url":"https://twitter.com/","potentialAction":{"@type":"SearchAction","query-input":"required name=search_term_string","target":{"@type":"EntryPoint","urlTemplate":"https://twitter.com/search?q={search_term_string}&ref_src=twcamp%5Eseo_searchbox%7Ctwsrc%5Eseo"}}}</script><meta http-equiv="origin-trial" content="AlpCmb40F5ZjDi9ZYe+wnr/V8MF+XmY41K4qUhoq+2mbepJTNd3q4CRqlACfnythEPZqcjryfAS1+ExS0FFRcA8AAABmeyJvcmlnaW4iOiJodHRwczovL3R3aXR0ZXIuY29tOjQ0MyIsImZlYXR1cmUiOiJMYXVuY2ggSGFuZGxlciIsImV4cGlyeSI6MTY1NTI1MTE5OSwiaXNTdWJkb21haW4iOnRydWV9" /><style>html,body{height: 100%;}::cue{white-space:normal}</style><meta content="article" property="og:type" /><meta content="https://twitter.com/UaSmart/status/934106870834454529" property="og:url" /><meta content="SmartUA (@UaSmart) on X" property="og:title" /><meta content="Y terminamos el dia con Cultura de empresa con @patoroco, @flopezluis, Katia, Angélica en @codemotion_es #codemotion2017" property="og:description" /><meta content="https://pbs.twimg.com/media/DPadOKpXcAIL-NW.jpg:large" property="og:image" /><meta content="Y terminamos el dia con Cultura de empresa con @patoroco, @flopezluis, Katia, Angélica en @codemotion_es #codemotion2017" name="description" /><meta content="SmartUA (@UaSmart) on X" name="title" /><meta content="twitter://status?id=934106870834454529" property="al:ios:url" /><meta content="333903271" property="al:ios:app_store_id" /><meta content="X" property="al:ios:app_name" /><meta content="twitter://status?id=934106870834454529" property="al:android:url" /><meta content="com.twitter.android" property="al:android:package" /><meta content="X" property="al:android:app_name" /></head><body style="background-color: #FFFFFF;"><div id="react-root" style="height:100%;display:flex;"></div>
2 changes: 2 additions & 0 deletions packages/metascraper-x/test/fixtures/post.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
<!DOCTYPE html><html dir="ltr" lang="en"><head><meta charset="utf-8" /><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0,viewport-fit=cover" /><link rel="preconnect" href="//abs.twimg.com" /><link rel="dns-prefetch" href="//abs.twimg.com" /><link rel="preconnect" href="//api.twitter.com" /><link rel="dns-prefetch" href="//api.twitter.com" /><link rel="preconnect" href="//api.x.com" /><link rel="dns-prefetch" href="//api.x.com" /><link rel="preconnect" href="//pbs.twimg.com" /><link rel="dns-prefetch" href="//pbs.twimg.com" /><link rel="preconnect" href="//t.co" /><link rel="dns-prefetch" href="//t.co" /><link rel="preconnect" href="//video.twimg.com" /><link rel="dns-prefetch" href="//video.twimg.com" /><meta http-equiv="onion-location" content="https://twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion/" /><meta property="fb:app_id" content="2231777543" /><meta content="X (formerly Twitter)" property="og:site_name" /><meta name="google-site-verification" content="600dQ0pZYsH2xOFt4hYmf5f5NpjCbWE_qk5Y04dErYM" /><meta name="facebook-domain-verification" content="x6sdcc8b5ju3bh8nbm59eswogvg6t1" /><meta name="mobile-web-app-capable" content="yes" /><meta name="apple-mobile-web-app-title" content="Twitter" /><meta name="apple-mobile-web-app-status-bar-style" content="white" /><link rel="search" type="application/opensearchdescription+xml" href="/os-x.xml" title="X">
<link rel="search" type="application/opensearchdescription+xml" href="/os-grok.xml" title="Grok"><link rel="manifest" href="/manifest.json" crossOrigin="use-credentials" /><link rel="search" type="application/opensearchdescription+xml" href="/os-x.xml" title="X"><link rel="search" type="application/opensearchdescription+xml" href="/os-grok.xml" title="Grok"><link rel="shortcut icon" href="//abs.twimg.com/favicons/twitter.3.ico"><meta name="theme-color" media="(prefers-color-scheme: light)" content="#FFFFFF" /><meta name="theme-color" media="(prefers-color-scheme: dark)" content="#000000" /><script type="application/ld+json">{"@context":"https://schema.org","@type":"WebSite","url":"https://twitter.com/","potentialAction":{"@type":"SearchAction","query-input":"required name=search_term_string","target":{"@type":"EntryPoint","urlTemplate":"https://twitter.com/search?q={search_term_string}&ref_src=twcamp%5Eseo_searchbox%7Ctwsrc%5Eseo"}}}</script><meta http-equiv="origin-trial" content="AlpCmb40F5ZjDi9ZYe+wnr/V8MF+XmY41K4qUhoq+2mbepJTNd3q4CRqlACfnythEPZqcjryfAS1+ExS0FFRcA8AAABmeyJvcmlnaW4iOiJodHRwczovL3R3aXR0ZXIuY29tOjQ0MyIsImZlYXR1cmUiOiJMYXVuY2ggSGFuZGxlciIsImV4cGlyeSI6MTY1NTI1MTE5OSwiaXNTdWJkb21haW4iOnRydWV9" /><style>html,body{height: 100%;}::cue{white-space:normal}</style><meta content="article" property="og:type" /><meta content="https://twitter.com/realDonaldTrump/status/1222907250383245320" property="og:url" /><meta content="Donald J. Trump (@realDonaldTrump) on X" property="og:title" /><meta content="“Schiff blasted for not focusing on California homeless.” @foxandfriends His District is in terrible shape. He is a corrupt pol who only dreams of the Impeachment Hoax. In my opinion he is mentally deranged!" property="og:description" /><meta content="https://pbs.twimg.com/profile_images/874276197357596672/kUuht00m_200x200.jpg" property="og:image" /><meta content="“Schiff blasted for not focusing on California homeless.” @foxandfriends His District is in terrible shape. He is a corrupt pol who only dreams of the Impeachment Hoax. In my opinion he is mentally deranged!" name="description" /><meta content="Donald J. Trump (@realDonaldTrump) on X" name="title" /><meta content="twitter://status?id=1222907250383245320" property="al:ios:url" /><meta content="333903271" property="al:ios:app_store_id" /><meta content="X" property="al:ios:app_name" /><meta content="twitter://status?id=1222907250383245320" property="al:android:url" /><meta content="com.twitter.android" property="al:android:package" /><meta content="X" property="al:android:app_name" /></head><body style="background-color: #FFFFFF;"><div id="react-root" style="height:100%;display:flex;"></div>
102 changes: 12 additions & 90 deletions packages/metascraper-x/test/fixtures/profile-video.html
View file

Large diffs are not rendered by default.

114 changes: 16 additions & 98 deletions packages/metascraper-x/test/fixtures/profile.html
View file

Large diffs are not rendered by default.

84 changes: 0 additions & 84 deletions packages/metascraper-x/test/fixtures/tweet-gif.html
View file

This file was deleted.

84 changes: 0 additions & 84 deletions packages/metascraper-x/test/fixtures/tweet-image.html
View file

This file was deleted.

26 changes: 0 additions & 26 deletions packages/metascraper-x/test/fixtures/tweet.html
View file

This file was deleted.

14 changes: 7 additions & 7 deletions packages/metascraper-x/test/index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,33 +44,33 @@ test('from a X profile resolving URLs', async t => {
t.snapshot(metadata)
})

test('from a X profile with tweets with video', async t => {
test('from a X profile with posts with video', async t => {
const url = 'https://x.com/javilop'
const html = await readFile(resolve(__dirname, 'fixtures/profile-video.html'))
const metascraper = createMetascraper()
const metadata = await metascraper({ url, html })
t.snapshot(metadata)
})

test('from a tweet', async t => {
test('from a post', async t => {
const url = 'https://x.com/realDonaldTrump/status/1222907250383245320'
const html = await readFile(resolve(__dirname, 'fixtures/tweet.html'))
const html = await readFile(resolve(__dirname, 'fixtures/post.html'))
const metascraper = createMetascraper()
const metadata = await metascraper({ url, html })
t.snapshot(metadata)
})

test('from a tweet with a gif', async t => {
test('from a post with a gif', async t => {
const url = 'https://x.com/Kikobeats/status/880139124791029763'
const html = await readFile(resolve(__dirname, 'fixtures/tweet-gif.html'))
const html = await readFile(resolve(__dirname, 'fixtures/post-gif.html'))
const metascraper = createMetascraper()
const metadata = await metascraper({ url, html })
t.snapshot(metadata)
})

test('from a tweet with an image', async t => {
test('from a post with an image', async t => {
const url = 'https://x.com/UaSmart/status/934106870834454529'
const html = await readFile(resolve(__dirname, 'fixtures/tweet-image.html'))
const html = await readFile(resolve(__dirname, 'fixtures/post-image.html'))
const metascraper = createMetascraper()
const metadata = await metascraper({ url, html })
t.snapshot(metadata)
Expand Down
88 changes: 76 additions & 12 deletions packages/metascraper-x/test/snapshots/index.js.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ Generated by [AVA](https://avajs.dev).
{
author: '#!/kiko/beats (Kikobeats)',
date: '2024-05-20T09:10:09.000Z',
date: null,
description: 'engineering ▲ @vercel; founder of https://t.co/4PQvCsVNsA https://t.co/fpiHwbEPBv https://t.co/IG8Qq0IDKi https://t.co/gblDRx1P9D https://t.co/SmoZi3hAhb https://t.co/Y0Uk1XU3Eu https://t.co/PAq3eTEhmI',
image: 'https://pbs.twimg.com/profile_images/1717583638991138816/4HvMeeps_400x400.jpg',
image: 'https://pbs.twimg.com/profile_images/1846292082501054464/oKUC44PF_400x400.jpg',
lang: 'en',
publisher: 'X',
title: '@kikobeats on X',
Expand All @@ -26,29 +26,93 @@ Generated by [AVA](https://avajs.dev).
{
author: '#!/kiko/beats (Kikobeats)',
date: '2024-05-20T09:10:09.000Z',
date: null,
description: 'engineering ▲ @vercel; founder of microlink.io teslahunt.io unavatar.io keyvhq.js.org osom.js.org browserless.js.org metascraper.js.org',
image: 'https://pbs.twimg.com/profile_images/1717583638991138816/4HvMeeps_400x400.jpg',
image: 'https://pbs.twimg.com/profile_images/1846292082501054464/oKUC44PF_400x400.jpg',
lang: 'en',
publisher: 'X',
title: '@Kikobeats on X',
url: 'https://x.com/Kikobeats',
video: null,
}

## from a X profile with posts with video

> Snapshot 1
{
author: 'Javi López ⛩️ (javilop)',
date: null,
description: 'Comparto tutoriales, herramientas y noticias de IA. Fundador @Magnific_AI 🔥 Guía IAs: https://t.co/JApwm5Tmfo 🗞️ Newsletter: https://t.co/tMELO1P8Wk',
image: 'https://pbs.twimg.com/profile_images/1581679886267301888/BHGZpOc6_400x400.jpg',
lang: 'en',
publisher: 'X',
title: '@javilop on X',
url: 'https://twitter.com/javilop',
video: null,
}

## from a post

> Snapshot 1
{
author: 'Donald J. Trump (realDonaldTrump)',
date: null,
description: '“Schiff blasted for not focusing on California homeless.” @foxandfriends His District is in terrible shape. He is a corrupt pol who only dreams of the Impeachment Hoax. In my opinion he is mentally deranged!',
image: 'https://pbs.twimg.com/profile_images/874276197357596672/kUuht00m_400x400.jpg',
lang: 'en',
publisher: 'X',
title: '@realDonaldTrump on X',
url: 'https://twitter.com/realDonaldTrump/status/1222907250383245320',
video: null,
}

## from a post with a gif

> Snapshot 1
{
author: '#!/kiko/beats (Kikobeats)',
date: null,
description: 'Experimenting with Clearbit API + Apple TV 3D Parallax https://t.co/Qsm163k4mJ',
image: 'https://pbs.twimg.com/tweet_video_thumb/DDbh3WCXYAAZfz9.jpg:large',
lang: 'en',
publisher: 'X',
title: '@Kikobeats on X',
url: 'https://twitter.com/Kikobeats/status/880139124791029763',
video: null,
}

## from a post with an image

> Snapshot 1
{
author: 'SmartUA (UaSmart)',
date: null,
description: 'Y terminamos el dia con Cultura de empresa con @patoroco, @flopezluis, Katia, Angélica en @codemotion_es #codemotion2017',
image: 'https://pbs.twimg.com/media/DPadOKpXcAIL-NW.jpg:large',
lang: 'en',
publisher: 'X',
title: '@UaSmart on X',
url: 'https://twitter.com/UaSmart/status/934106870834454529',
video: null,
}

## from a X profile with tweets with video

> Snapshot 1
{
author: 'Javi López ⛩️ (javilop)',
date: '2024-05-20T09:35:21.000Z',
date: null,
description: 'Comparto tutoriales, herramientas y noticias de IA. Fundador @Magnific_AI 🔥 Guía IAs: https://t.co/JApwm5Tmfo 🗞️ Newsletter: https://t.co/tMELO1P8Wk',
image: 'https://pbs.twimg.com/profile_images/1581679886267301888/BHGZpOc6_400x400.jpg',
lang: 'en',
publisher: 'X',
title: '@javilop on X',
url: 'https://x.com/javilop/',
url: 'https://twitter.com/javilop',
video: null,
}

Expand All @@ -58,13 +122,13 @@ Generated by [AVA](https://avajs.dev).
{
author: 'Donald J. Trump (realDonaldTrump)',
date: '2024-05-20T09:39:36.000Z',
date: null,
description: '“Schiff blasted for not focusing on California homeless.” @foxandfriends His District is in terrible shape. He is a corrupt pol who only dreams of the Impeachment Hoax. In my opinion he is mentally deranged!',
image: 'https://pbs.twimg.com/profile_images/874276197357596672/kUuht00m_400x400.jpg',
lang: 'en',
publisher: 'X',
title: '@realDonaldTrump on X',
url: 'https://x.com/realDonaldTrump/status/1222907250383245320',
url: 'https://twitter.com/realDonaldTrump/status/1222907250383245320',
video: null,
}

Expand All @@ -74,13 +138,13 @@ Generated by [AVA](https://avajs.dev).
{
author: '#!/kiko/beats (Kikobeats)',
date: '2024-05-20T09:40:45.000Z',
date: null,
description: 'Experimenting with Clearbit API + Apple TV 3D Parallax https://t.co/Qsm163k4mJ',
image: 'https://pbs.twimg.com/tweet_video_thumb/DDbh3WCXYAAZfz9.jpg:large',
lang: 'en',
publisher: 'X',
title: '@Kikobeats on X',
url: 'https://x.com/Kikobeats/status/880139124791029763',
url: 'https://twitter.com/Kikobeats/status/880139124791029763',
video: null,
}

Expand All @@ -90,12 +154,12 @@ Generated by [AVA](https://avajs.dev).
{
author: 'SmartUA (UaSmart)',
date: '2024-05-20T09:48:26.000Z',
date: null,
description: 'Y terminamos el dia con Cultura de empresa con @patoroco, @flopezluis, Katia, Angélica en @codemotion_es #codemotion2017',
image: 'https://pbs.twimg.com/media/DPadOKpXcAIL-NW.jpg:large',
lang: 'en',
publisher: 'X',
title: '@UaSmart on X',
url: 'https://x.com/UaSmart/status/934106870834454529',
url: 'https://twitter.com/UaSmart/status/934106870834454529',
video: null,
}
Binary file modified packages/metascraper-x/test/snapshots/index.js.snap
View file
Binary file not shown.

0 comments on commit d126e2e

Please sign in to comment.