Merge branch 'iv-org:master' into optional-disable-api-features

This commit is contained in:
Richard Lora 2024-11-11 15:18:12 -05:00 committed by GitHub
commit b4f8a6726b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
77 changed files with 1149 additions and 578 deletions

View File

@ -38,6 +38,9 @@ Style/RedundantBegin:
Style/RedundantReturn:
Enabled: false
Style/RedundantNext:
Enabled: false
Style/ParenthesesAroundCondition:
Enabled: false

2
.github/CODEOWNERS vendored
View File

@ -6,7 +6,7 @@ docker/ @unixfox
kubernetes/ @unixfox
README.md @thefrenchghosty
config/config.example.yml @thefrenchghosty @SamantazFox @unixfox
config/config.example.yml @SamantazFox @unixfox
scripts/ @syeopite
shards.lock @syeopite

View File

@ -38,10 +38,11 @@ jobs:
matrix:
stable: [true]
crystal:
- 1.9.2
- 1.10.1
- 1.11.2
- 1.12.1
- 1.13.2
- 1.14.0
include:
- crystal: nightly
stable: false
@ -51,6 +52,11 @@ jobs:
with:
submodules: true
- name: Install required APT packages
run: |
sudo apt install -y libsqlite3-dev
shell: bash
- name: Install Crystal
uses: crystal-lang/install-crystal@v1.8.0
with:
@ -59,7 +65,9 @@ jobs:
- name: Cache Shards
uses: actions/cache@v3
with:
path: ./lib
path: |
./lib
./bin
key: shards-${{ hashFiles('shard.lock') }}
- name: Install Shards
@ -71,14 +79,6 @@ jobs:
- name: Run tests
run: crystal spec
- name: Run lint
run: |
if ! crystal tool format --check; then
crystal tool format
git diff
exit 1
fi
- name: Build
run: crystal build --warnings all --error-on-warnings --error-trace src/invidious.cr
@ -124,8 +124,12 @@ jobs:
- name: Test Docker
run: while curl -Isf http://localhost:3000; do sleep 1; done
ameba_lint:
lint:
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v4
with:
@ -145,7 +149,18 @@ jobs:
key: shards-${{ hashFiles('shard.lock') }}
- name: Install Shards
run: shards install
run: |
if ! shards check; then
shards install
fi
- name: Check Crystal formatter compliance
run: |
if ! crystal tool format --check; then
crystal tool format
git diff
exit 1
fi
- name: Run Ameba linter
run: bin/ameba

View File

@ -13,14 +13,11 @@ jobs:
- uses: actions/stale@v8
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
days-before-stale: 365
days-before-pr-stale: 90
days-before-close: 30
exempt-pr-labels: blocked,exempt-stale
days-before-stale: 730
days-before-pr-stale: -1
days-before-close: 60
stale-issue-message: 'This issue has been automatically marked as stale and will be closed in 30 days because it has not had recent activity and is much likely outdated. If you think this issue is still relevant and applicable, you just have to post a comment and it will be unmarked.'
stale-pr-message: 'This pull request has been automatically marked as stale and will be closed in 30 days because it has not had recent activity and is much likely abandoned or outdated. If you think this pull request is still relevant and applicable, you just have to post a comment and it will be unmarked.'
stale-issue-label: "stale"
stale-pr-label: "stale"
ascending: true
# Never mark feature requests/enhancements as stale
exempt-issue-labels: "feature-request,enhancement,exempt-stale"
# Exempt the following types of issues from being staled
exempt-issue-labels: "feature-request,enhancement,discussion,exempt-stale"

View File

@ -1,5 +1,149 @@
# CHANGELOG
## vX.Y.0 (future)
## v2.20241110.0
### Wrap-up
This release is most importantly here to fix to the annoying "Youtube API returned error 400"
error that prevented all channel pages from loading.
If you're updating from the previous release, it provides no improvements on the ability to play
videos. If updating from a commit in-between release, it removes the "Please sign in" error caused
by a previous attempt at restoring video playback on large instances.
In the preferences, a new option allows for control of video preload. When enabled, this option
tells the browser to load the video as soon as the page is loaded (this used to be the default).
When disabled, the video starts loading only when the "play" button is pressed.
New interface languages available: Bulgarian, Welsh and Lombard
New dependency required: `tzdata`.
An HTTP proxy can be configured directly in Invidious, if needed. \
**NOTE:** In that case, it is recommended to comment out `force_resolve`.
### New features & important changes
#### For users
* Channels: Fix "Youtube API returned error 400" error preventing channel pages from loading
* Channels: Shorts can now be sorted by "newest", "oldest" and "popular"
* Preferences: Addition of the new "preload" option
* New interface languages available: Bulgarian, Welsh and Lombard
* Added "Filipino (auto-generated)" to the list of caption languages available
* Lots of new translations from Weblate
#### For instance owners
* Allow the configuration of an HTTP proxy to talk to Youtube
* Invidious tries to reconnect to `inv_sig_helper` if the socket is closed
* The instance list is downloaded in the background to improve redirection speed
* New `colorize_logs` option makes each log level a different color
#### For developpers
* `/api/v1/channels/{id}/shorts` now supports the `sort-by` parameter with the following values:
`newest`, `oldest` and `popular`
* Older `/api/v1/channels/xyz/{id}` (tab name before UCID) were removed
* API/Search: New video metadata available: `isNew`, `is4k`, `is8k`, `isVr180`, `isVr360`,
`is3d` and `hasCaptions`
### Bugs fixed
#### User-side
* Channels: The second page of shorts now loads as expected
* Channels: Fixed intermittent empty "playlists" tab
* Search: Fixed `youtu.be` URLs not being properly redirected to the watch page
* Fixed `DB::MappingException` error on the subscriptions feed (due to missing `tzdata` in docker)
* Switching to another instance is much faster
* Fixed an "invalid byte sequence" error when subscribing to a playlist
* Videos: Playback URLs were sometimes broken when cached and `inv_sig_helper` was used
#### For instance owners
* Fix `force_resolve` being ignored in some cases
#### API
* API/Videos: Fixed `live_now` and `premiere_timestamp` sometimes not having the right values
### Full list of pull requests merged since the last release (newest first)
* API: Add "sort_by" parameter to channels/shorts endpoint ([#5071], thanks @iBicha)
* Docker: Install tzdata in Dockerfile ([#5070], by @SamantazFox)
* Videos: Stop using TVHTML5_SIMPLY_EMBEDDED_PLAYER ([#5063], thanks @unixfox)
* Routing: Deprecate old channel API routes ([#5045], by @SamantazFox)
* Videos: use WEB client instead of WEB CREATOR ([#4984], thanks @unixfox)
* Parsers: Fix parsing live_now and premiere_timestamp ([#4934], thanks @absidue)
* Stale bot updates ([#5060], thanks @syeopite)
* Channels: Fix "Youtube API returned error 400" ([#5059], by @SamantazFox)
* Channels: Fix for live videos ([#5027], thanks @iBicha)
* Locales: Add Bulgarian, Welsh and Lombard to the list ([#5046], by @SamantazFox)
* Shards: Update database dependencies ([#5034], by @SamantazFox)
* Logger: Add color support for different log levels ([#4931], thanks @Fijxu)
* Fix named arg syntax when passing force_resolve ([#4754], thanks @syeopite)
* Use make_client instead of calling HTTP::Client ([#4709], thanks @syeopite)
* Add "Filipino (auto-generated)" to the list of caption languages ([#4995], by @SamantazFox)
* Makefile: Add MT option to enable the 'preview_mt' flag ([#4993], by @SamantazFox)
* SigHelper: Reconnect to signature helper ([#4991], thanks @Fijxu)
* Fix player menus hiding onHover ready ([#4750], thanks @giacomocerquone)
* Use connection pools when requesting images from YouTube ([#4326], thanks @syeopite)
* Add support for using Invidious through a HTTP Proxy ([#4270], thanks @syeopite)
* Search: Fix 'youtu.be' URLs in sanitizer ([#4894], by @SamantazFox)
* Ameba: Disable Style/RedundantNext rule ([#4888], thanks @syeopite)
* Playlists: Fix 'invalid byte sequence' error when subscribing ([#4887], thanks @DmitrySandalov)
* Parse more metadata badges for SearchVideos ([#4863], thanks @ChunkyProgrammer)
* Translations update from Hosted Weblate ([#4862], thanks to our many translators)
* Videos: Convert URL before putting result into cache ([#4850], by @SamantazFox)
* HTML: Add error message to "search issues on GitHub" link ([#4652], thanks @tracedgod)
* Preferences: Add option to control preloading of video data ([#4122], thanks @Nerdmind)
* Performance: Improve speed of automatic instance redirection ([#4193], thanks @syeopite)
* Remove myself from CODEOWNERS on the config file ([#4942], by @TheFrenchGhosty)
* Update latest version WEB_CREATOR + fix comment web embed ([#4930], thanks @unixfox)
* use WEB_CREATOR when po_token with WEB_EMBED as a fallback ([#4928], thanks @unixfox)
* Revert "use web screen embed for fixing potoken functionality"
* use web screen embed for fixing potoken functionality ([#4923], thanks @unixfox)
[#4122]: https://github.com/iv-org/invidious/pull/4122
[#4193]: https://github.com/iv-org/invidious/pull/4193
[#4270]: https://github.com/iv-org/invidious/pull/4270
[#4326]: https://github.com/iv-org/invidious/pull/4326
[#4652]: https://github.com/iv-org/invidious/pull/4652
[#4709]: https://github.com/iv-org/invidious/pull/4709
[#4750]: https://github.com/iv-org/invidious/pull/4750
[#4754]: https://github.com/iv-org/invidious/pull/4754
[#4850]: https://github.com/iv-org/invidious/pull/4850
[#4862]: https://github.com/iv-org/invidious/pull/4862
[#4863]: https://github.com/iv-org/invidious/pull/4863
[#4887]: https://github.com/iv-org/invidious/pull/4887
[#4888]: https://github.com/iv-org/invidious/pull/4888
[#4894]: https://github.com/iv-org/invidious/pull/4894
[#4923]: https://github.com/iv-org/invidious/pull/4923
[#4928]: https://github.com/iv-org/invidious/pull/4928
[#4930]: https://github.com/iv-org/invidious/pull/4930
[#4931]: https://github.com/iv-org/invidious/pull/4931
[#4934]: https://github.com/iv-org/invidious/pull/4934
[#4942]: https://github.com/iv-org/invidious/pull/4942
[#4984]: https://github.com/iv-org/invidious/pull/4984
[#4991]: https://github.com/iv-org/invidious/pull/4991
[#4993]: https://github.com/iv-org/invidious/pull/4993
[#4995]: https://github.com/iv-org/invidious/pull/4995
[#5027]: https://github.com/iv-org/invidious/pull/5027
[#5034]: https://github.com/iv-org/invidious/pull/5034
[#5045]: https://github.com/iv-org/invidious/pull/5045
[#5046]: https://github.com/iv-org/invidious/pull/5046
[#5059]: https://github.com/iv-org/invidious/pull/5059
[#5060]: https://github.com/iv-org/invidious/pull/5060
[#5063]: https://github.com/iv-org/invidious/pull/5063
[#5070]: https://github.com/iv-org/invidious/pull/5070
[#5071]: https://github.com/iv-org/invidious/pull/5071
## v2.20240825.2 (2024-08-26)

View File

@ -7,6 +7,11 @@ STATIC := 0
NO_DBG_SYMBOLS := 0
# Enable multi-threading.
# Warning: Experimental feature!!
# invidious is not stable when MT is enabled.
MT := 0
FLAGS ?=
@ -19,6 +24,10 @@ ifeq ($(STATIC), 1)
FLAGS += --static
endif
ifeq ($(MT), 1)
FLAGS += -Dpreview_mt
endif
ifeq ($(NO_DBG_SYMBOLS), 1)
FLAGS += --no-debug

View File

@ -68,6 +68,7 @@
.video-js.player-style-youtube .vjs-menu-button-popup .vjs-menu {
margin-bottom: 2em;
padding-top: 2em
}
.video-js.player-style-youtube .vjs-progress-control .vjs-progress-holder, .video-js.player-style-youtube .vjs-progress-control {height: 5px;

View File

@ -3,7 +3,6 @@ var player_data = JSON.parse(document.getElementById('player_data').textContent)
var video_data = JSON.parse(document.getElementById('video_data').textContent);
var options = {
preload: 'auto',
liveui: true,
playbackRates: [0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0],
controlBar: {

View File

@ -173,6 +173,17 @@ https_only: false
##
#force_resolve:
##
## Configuration for using a HTTP proxy
##
## If unset, then no HTTP proxy will be used.
##
http_proxy:
user:
password:
host:
port:
##
## Use Innertube's transcripts API instead of timedtext for closed captions
@ -222,6 +233,17 @@ https_only: false
##
#log_level: Info
##
## Enables colors in logs. Useful for debugging purposes
## This is overridden if "-k" or "--colorize"
## are passed on the command line.
## Colors are also disabled if the environment variable
## NO_COLOR is present and has any value
##
## Accepted values: true, false
## Default: true
##
#colorize_logs: false
# -----------------------------
# Features
@ -708,6 +730,22 @@ default_user_preferences:
# Video player behavior
# -----------------------------
##
## This option controls the value of the HTML5 <video> element's
## "preload" attribute.
##
## If set to 'false', no video data will be loaded until the user
## explicitly starts the video by clicking the "Play" button.
## If set to 'true', the web browser will buffer some video data
## while the page is loading.
##
## See: https://www.w3schools.com/tags/att_video_preload.asp
##
## Accepted values: true, false
## Default: true
##
#preload: true
##
## Automatically play videos on page load.
##

View File

@ -1,4 +1,4 @@
FROM crystallang/crystal:1.12.1-alpine AS builder
FROM crystallang/crystal:1.12.2-alpine AS builder
RUN apk add --no-cache sqlite-static yaml-static
@ -32,8 +32,8 @@ RUN if [[ "${release}" == 1 ]] ; then \
--link-flags "-lxml2 -llzma"; \
fi
FROM alpine:3.18
RUN apk add --no-cache rsvg-convert ttf-opensans tini
FROM alpine:3.20
RUN apk add --no-cache rsvg-convert ttf-opensans tini tzdata
WORKDIR /invidious
RUN addgroup -g 1000 -S invidious && \
adduser -u 1000 -S invidious -G invidious

View File

@ -1,5 +1,6 @@
FROM alpine:3.19 AS builder
RUN apk add --no-cache 'crystal=1.10.1-r0' shards sqlite-static yaml-static yaml-dev libxml2-static zlib-static openssl-libs-static openssl-dev musl-dev xz-static
FROM alpine:3.20 AS builder
RUN apk add --no-cache 'crystal=1.12.2-r0' shards sqlite-static yaml-static yaml-dev libxml2-static \
zlib-static openssl-libs-static openssl-dev musl-dev xz-static
ARG release
@ -32,8 +33,8 @@ RUN if [[ "${release}" == 1 ]] ; then \
--link-flags "-lxml2 -llzma"; \
fi
FROM alpine:3.18
RUN apk add --no-cache rsvg-convert ttf-opensans tini
FROM alpine:3.20
RUN apk add --no-cache rsvg-convert ttf-opensans tini tzdata
WORKDIR /invidious
RUN addgroup -g 1000 -S invidious && \
adduser -u 1000 -S invidious -G invidious

View File

@ -47,6 +47,7 @@
"Preferences": "Einstellungen",
"preferences_category_player": "Wiedergabeeinstellungen",
"preferences_video_loop_label": "Immer wiederholen: ",
"preferences_preload_label": "Videodaten vorladen: ",
"preferences_autoplay_label": "Automatisch abspielen: ",
"preferences_continue_label": "Immer automatisch nächstes Video abspielen: ",
"preferences_continue_autoplay_label": "Nächstes Video automatisch abspielen: ",
@ -322,7 +323,7 @@
"channel_tab_community_label": "Gemeinschaft",
"search_filters_sort_option_relevance": "Relevanz",
"search_filters_sort_option_rating": "Bewertung",
"search_filters_sort_option_date": "Datum",
"search_filters_sort_option_date": "Hochladedatum",
"search_filters_sort_option_views": "Aufrufe",
"search_filters_type_label": "Inhaltstyp",
"search_filters_duration_label": "Dauer",
@ -493,5 +494,8 @@
"Add to playlist": "Einer Wiedergabeliste hinzufügen",
"Search for videos": "Nach Videos suchen",
"toggle_theme": "Thema wechseln",
"Add to playlist: ": "Einer Wiedergabeliste hinzufügen: "
"Add to playlist: ": "Einer Wiedergabeliste hinzufügen: ",
"carousel_go_to": "Zu Folie `x` gehen",
"carousel_slide": "Folie {{current}} von {{total}}",
"carousel_skip": "Karussell überspringen"
}

View File

@ -489,5 +489,10 @@
"search_filters_date_label": "Ημερομηνία αναφόρτωσης",
"Search for videos": "Αναζήτηση βίντεο",
"The Popular feed has been disabled by the administrator.": "Η δημοφιλής ροή έχει απενεργοποιηθεί από τον διαχειριστή.",
"Answer": "Απάντηση"
"Answer": "Απάντηση",
"Add to playlist": "Λίιστα αναπαραγωγής",
"Add to playlist: ": "Λίστα αναπαραγωγής: ",
"carousel_slide": "Εικόνα {{current}}απο {{total}}",
"carousel_go_to": "Πήγαινε στην εικόνα`x`",
"toggle_theme": "Αλλαγή θέματος"
}

View File

@ -71,6 +71,7 @@
"Preferences": "Preferences",
"preferences_category_player": "Player preferences",
"preferences_video_loop_label": "Always loop: ",
"preferences_preload_label": "Preload video data: ",
"preferences_autoplay_label": "Autoplay: ",
"preferences_continue_label": "Play next by default: ",
"preferences_continue_autoplay_label": "Autoplay next video: ",
@ -285,6 +286,7 @@
"Esperanto": "Esperanto",
"Estonian": "Estonian",
"Filipino": "Filipino",
"Filipino (auto-generated)": "Filipino (auto-generated)",
"Finnish": "Finnish",
"French": "French",
"French (auto-generated)": "French (auto-generated)",
@ -422,7 +424,7 @@
"search_filters_title": "Filters",
"search_filters_date_label": "Upload date",
"search_filters_date_option_none": "Any date",
"search_filters_date_option_hour": "Last Hour",
"search_filters_date_option_hour": "Last hour",
"search_filters_date_option_today": "Today",
"search_filters_date_option_week": "This week",
"search_filters_date_option_month": "This month",
@ -454,7 +456,7 @@
"search_filters_sort_label": "Sort By",
"search_filters_sort_option_relevance": "Relevance",
"search_filters_sort_option_rating": "Rating",
"search_filters_sort_option_date": "Upload Date",
"search_filters_sort_option_date": "Upload date",
"search_filters_sort_option_views": "View count",
"search_filters_apply_button": "Apply selected filters",
"Current version: ": "Current version: ",

View File

@ -478,7 +478,7 @@
"tokens_count_0": "{{count}} token",
"tokens_count_1": "{{count}} tokens",
"tokens_count_2": "{{count}} tokens",
"search_message_use_another_instance": " También puede <a href=\"`x`\">buscar en otra instancia</a>.",
"search_message_use_another_instance": "También puedes <a href=\"`x`\">buscar en otra instancia</a>.",
"Popular enabled: ": "¿Habilitar la sección popular? ",
"error_video_not_in_playlist": "El video que solicitaste no existe en esta lista de reproducción. <a href=\"`x`\">Haz clic aquí para acceder a la página de inicio de la lista de reproducción.</a>",
"channel_tab_streams_label": "Directos",

View File

@ -360,7 +360,7 @@
"search_filters_duration_label": "مدت",
"search_filters_features_label": "ویژگی‌ها",
"search_filters_sort_label": "به ترتیب",
"search_filters_date_option_hour": "یک ساعت گذشته",
"search_filters_date_option_hour": "ساعت گذشته",
"search_filters_date_option_today": "امروز",
"search_filters_date_option_week": "این هفته",
"search_filters_date_option_month": "این ماه",
@ -461,7 +461,7 @@
"Song: ": "آهنگ: ",
"Channel Sponsor": "اسپانسر کانال",
"Standard YouTube license": "پروانه استاندارد YouTube",
"search_message_use_another_instance": " شما همچنین می‌توانید <a href=\"`x`\">در نمونه دیگر هم جستجو کنید</a>.",
"search_message_use_another_instance": "همچنین می‌توانید <a href=\"`x`\">در نمونه‌ای دیگر هم جست‌وجو کنید</a>.",
"Download is disabled": "دریافت غیرفعال است",
"crash_page_before_reporting": "پیش از گزارش ایراد، مطمئنید شوید که:",
"playlist_button_add_items": "افزودن ویدیو",

View File

@ -449,24 +449,24 @@
"Cantonese (Hong Kong)": "Kantonski (Hong Kong)",
"Chinese": "Kineski",
"Chinese (Taiwan)": "Kineski (Tajvan)",
"Dutch (auto-generated)": "Nizozemski (automatski generiran)",
"French (auto-generated)": "Francuski (automatski generiran)",
"Indonesian (auto-generated)": "Indonezijski (automatski generiran)",
"Dutch (auto-generated)": "Nizozemski (automatski generirano)",
"French (auto-generated)": "Francuski (automatski generirano)",
"Indonesian (auto-generated)": "Indonezijski (automatski generirano)",
"Interlingue": "Interlingua",
"Japanese (auto-generated)": "Japanski (automatski generiran)",
"Russian (auto-generated)": "Ruski (automatski generiran)",
"Turkish (auto-generated)": "Turski (automatski generiran)",
"Vietnamese (auto-generated)": "Vijetnamski (automatski generiran)",
"Japanese (auto-generated)": "Japanski (automatski generirano)",
"Russian (auto-generated)": "Ruski (automatski generirano)",
"Turkish (auto-generated)": "Turski (automatski generirano)",
"Vietnamese (auto-generated)": "Vijetnamski (automatski generirano)",
"Spanish (Spain)": "Španjolski (Španjolska)",
"Italian (auto-generated)": "Talijanski (automatski generiran)",
"Italian (auto-generated)": "Talijanski (automatski generirano)",
"Portuguese (Brazil)": "Portugalski (Brazil)",
"Spanish (Mexico)": "Španjolski (Meksiko)",
"German (auto-generated)": "Njemački (automatski generiran)",
"German (auto-generated)": "Njemački (automatski generirano)",
"Chinese (China)": "Kineski (Kina)",
"Chinese (Hong Kong)": "Kineski (Hong Kong)",
"Korean (auto-generated)": "Korejski (automatski generiran)",
"Portuguese (auto-generated)": "Portugalski (automatski generiran)",
"Spanish (auto-generated)": "Španjolski (automatski generiran)",
"Korean (auto-generated)": "Korejski (automatski generirano)",
"Portuguese (auto-generated)": "Portugalski (automatski generirano)",
"Spanish (auto-generated)": "Španjolski (automatski generirano)",
"preferences_watch_history_label": "Aktiviraj povijest gledanja: ",
"search_filters_title": "Filtri",
"search_filters_date_option_none": "Bilo koji datum",

View File

@ -7,7 +7,7 @@
"invidious": "Invidious",
"Image CAPTCHA": "Imagine CAPTCHA",
"newest": "plus nove",
"generic_button_save": "Salvar",
"generic_button_save": "Salveguardar",
"Dark mode: ": "Modo obscur: ",
"preferences_dark_mode_label": "Thema: ",
"preferences_category_subscription": "Preferentias de subscription",
@ -23,7 +23,7 @@
"light": "clar",
"No": "Non",
"youtube": "YouTube",
"LIVE": "IN DIRECTE",
"LIVE": "IN DIRECTO",
"reddit": "Reddit",
"preferences_category_player": "Preferentias de reproductor",
"Preferences": "Preferentias",

View File

@ -363,7 +363,7 @@
"search_filters_features_option_location": "場所",
"search_filters_features_option_hdr": "HDR",
"Current version: ": "現在のバージョン: ",
"next_steps_error_message": "以下をお試しください: ",
"next_steps_error_message": "以下をお試しください: ",
"next_steps_error_message_refresh": "再読み込み",
"next_steps_error_message_go_to_youtube": "YouTubeを開く",
"search_filters_duration_option_short": "4分未満",
@ -396,7 +396,7 @@
"download_subtitles": "字幕 - `x` (.vtt)",
"search_filters_features_option_purchased": "購入済み",
"preferences_quality_option_dash": "DASH (適応的画質)",
"preferences_quality_dash_option_worst": "最",
"preferences_quality_dash_option_worst": "最",
"preferences_quality_dash_option_best": "最高",
"videoinfo_started_streaming_x_ago": "`x`前に配信を開始",
"videoinfo_watch_on_youTube": "YouTubeで視聴",

View File

@ -18,8 +18,8 @@
"preferences_related_videos_label": "관련 동영상 보기: ",
"Fallback captions: ": "대체 자막: ",
"preferences_captions_label": "기본 자막: ",
"reddit": "Reddit",
"youtube": "YouTube",
"reddit": "레딧",
"youtube": "유튜브",
"preferences_comments_label": "기본 댓글: ",
"preferences_volume_label": "플레이어 볼륨: ",
"preferences_quality_label": "선호하는 비디오 품질: ",
@ -48,7 +48,7 @@
"An alternative front-end to YouTube": "유튜브의 프론트엔드 대안",
"History": "시청 기록",
"Delete account?": "계정을 삭제 하시겠습니까?",
"Export data as JSON": "JSON으로 데이터 내보내기",
"Export data as JSON": "인비디어스 데이터 내보내기 (.json)",
"Export subscriptions as OPML (for NewPipe & FreeTube)": "OPML로 구독 내보내기 (뉴파이프 및 프리튜브)",
"Export subscriptions as OPML": "OPML로 구독 내보내기",
"Export": "내보내기",
@ -78,10 +78,10 @@
"Subscribe": "구독",
"Unsubscribe": "구독 취소",
"LIVE": "실시간",
"generic_views_count_0": "조회수 {{count}}회",
"generic_videos_count_0": "동영상 {{count}}개",
"generic_playlists_count_0": "재생목록 {{count}}개",
"generic_subscribers_count_0": "구독자 {{count}}명",
"generic_views_count_0": "{{count}}",
"generic_videos_count_0": "{{count}} 동영상",
"generic_playlists_count_0": "{{count}} 재생목록",
"generic_subscribers_count_0": "{{count}} 구독자",
"generic_subscriptions_count_0": "{{count}} 구독",
"search_filters_type_option_playlist": "재생목록",
"Korean": "한국어",
@ -109,14 +109,14 @@
"This channel does not exist.": "이 채널은 존재하지 않습니다.",
"Deleted or invalid channel": "삭제되었거나 더 이상 존재하지 않는 채널",
"channel:`x`": "채널:`x`",
"Show replies": "댓글 보기",
"Show replies": "댓글 보기",
"Hide replies": "댓글 숨기기",
"Incorrect password": "잘못된 비밀번호",
"License: ": "라이선스: ",
"Genre: ": "장르: ",
"Editing playlist `x`": "재생목록 `x` 수정하기",
"Playlist privacy": "재생목록 공개 범위",
"Watch on YouTube": "YouTube에서 보기",
"Watch on YouTube": "유튜브에서 보기",
"Show less": "간략히",
"Show more": "더보기",
"Title": "제목",
@ -125,7 +125,7 @@
"Delete playlist": "재생목록 삭제",
"Delete playlist `x`?": "재생목록 `x` 를 삭제하시겠습니까?",
"Updated `x` ago": "`x` 전에 업데이트됨",
"Released under the AGPLv3 on Github.": "GitHub에 AGPLv3 으로 배포됩니다.",
"Released under the AGPLv3 on Github.": "깃허브에 AGPLv3 으로 배포됩니다.",
"View all playlists": "모든 재생목록 보기",
"Private": "비공개",
"Unlisted": "목록에 없음",
@ -135,12 +135,12 @@
"Source available here.": "소스는 여기에서 사용할 수 있습니다.",
"Log out": "로그아웃",
"search": "검색",
"subscriptions_unseen_notifs_count_0": "읽지 않은 알림 {{count}}개",
"subscriptions_unseen_notifs_count_0": "{{count}} 읽지 않은 알림",
"Subscriptions": "구독",
"revoke": "철회",
"unsubscribe": "구독 취소",
"Import/export": "가져오기/내보내기",
"tokens_count_0": "토큰 {{count}}개",
"tokens_count_0": "{{count}} 토큰",
"Token": "토큰",
"Token manager": "토큰 관리자",
"Subscription manager": "구독 관리자",
@ -163,7 +163,7 @@
"Clear watch history": "시청 기록 지우기",
"preferences_category_data": "데이터 설정",
"`x` is live": "`x` 이(가) 라이브 중입니다",
"`x` uploaded a video": "`x` 이(가) 동영상을 게시했습니다",
"`x` uploaded a video": "`x` 동영상 게시됨",
"Enable web notifications": "웹 알림 활성화",
"preferences_notifications_only_label": "알림만 표시 (있는 경우): ",
"preferences_unseen_only_label": "시청하지 않은 것만 표시: ",
@ -241,7 +241,7 @@
"Could not create mix.": "믹스를 생성할 수 없습니다.",
"`x` ago": "`x` 전",
"comments_view_x_replies_0": "답글 {{count}}개 보기",
"View Reddit comments": "Reddit 댓글 보기",
"View Reddit comments": "레딧 댓글 보기",
"Engagement: ": "약속: ",
"Wilson score: ": "Wilson Score: ",
"Family friendly? ": "전연령 영상입니까? ",
@ -267,8 +267,8 @@
"Bulgarian": "불가리아어",
"Bosnian": "보스니아어",
"Belarusian": "벨라루스어",
"View more comments on Reddit": "Reddit에서 댓글 더 보기",
"View YouTube comments": "YouTube 댓글 보기",
"View more comments on Reddit": "레딧에서 댓글 더 보기",
"View YouTube comments": "유튜브 댓글 보기",
"Hi! Looks like you have JavaScript turned off. Click here to view comments, keep in mind they may take a bit longer to load.": "자바스크립트가 꺼져 있는 것 같습니다! 댓글을 보려면 여기를 클릭하세요. 댓글을 로드하는 데 시간이 조금 더 걸릴 수 있습니다.",
"Shared `x`": "`x` 업로드",
"Whitelisted regions: ": "차단되지 않은 지역: ",
@ -289,7 +289,7 @@
"Empty playlist": "재생목록 비어 있음",
"Show annotations": "주석 보이기",
"Hide annotations": "주석 숨기기",
"Switch Invidious Instance": "Invidious 인스턴스 변경",
"Switch Invidious Instance": "인비디어스 인스턴스 변경",
"Spanish": "스페인어",
"Southern Sotho": "소토어",
"Somali": "소말리어",
@ -329,7 +329,7 @@
"Swedish": "스웨덴어",
"Spanish (Latin America)": "스페인어 (라틴 아메리카)",
"comments_points_count_0": "{{count}} 포인트",
"Invidious Private Feed for `x`": "`x` 에 대한 Invidious 비공개 피드",
"Invidious Private Feed for `x`": "`x` 에 대한 인비디어스 비공개 피드",
"Premieres `x`": "최초 공개 `x`",
"Premieres in `x`": "`x` 후 최초 공개",
"next_steps_error_message": "다음 방법을 시도해 보세요: ",
@ -408,7 +408,7 @@
"preferences_quality_dash_option_1080p": "1080p",
"preferences_quality_dash_option_worst": "최저",
"preferences_watch_history_label": "시청 기록 저장: ",
"invidious": "Invidious",
"invidious": "인비디어스",
"preferences_quality_option_small": "낮음",
"preferences_quality_dash_option_auto": "자동",
"preferences_quality_dash_option_480p": "480p",
@ -453,7 +453,7 @@
"channel_tab_streams_label": "실시간 스트리밍",
"channel_tab_channels_label": "채널",
"channel_tab_playlists_label": "재생목록",
"Standard YouTube license": "표준 YouTube 라이선스",
"Standard YouTube license": "표준 유튜브 라이선스",
"Song: ": "제목: ",
"Channel Sponsor": "채널 스폰서",
"Album: ": "앨범: ",

View File

@ -322,13 +322,13 @@
"channel_tab_community_label": "Gemenskap",
"search_filters_sort_option_relevance": "relevans",
"search_filters_sort_option_rating": "vurdering",
"search_filters_sort_option_date": "dato",
"search_filters_sort_option_date": "Opplastingsdato",
"search_filters_sort_option_views": "visninger",
"search_filters_type_label": "innholdstype",
"search_filters_duration_label": "varighet",
"search_filters_features_label": "funksjoner",
"search_filters_sort_label": "sorter",
"search_filters_date_option_hour": "time",
"search_filters_date_option_hour": "Siste time",
"search_filters_date_option_today": "i dag",
"search_filters_date_option_week": "uke",
"search_filters_date_option_month": "måned",
@ -494,5 +494,7 @@
"carousel_slide": "Lysark {{current}} av {{total}}",
"carousel_skip": "Hopp over karusellen",
"Add to playlist": "Legg til i spilleliste",
"Add to playlist: ": "Legg til i spilleliste: "
"Add to playlist: ": "Legg til i spilleliste: ",
"The Popular feed has been disabled by the administrator.": "Populært-kilden er koblet ut av administratoren.",
"toggle_theme": "Endre utseende"
}

View File

@ -317,13 +317,13 @@
"channel_tab_community_label": "Gemeenschap",
"search_filters_sort_option_relevance": "relevantie",
"search_filters_sort_option_rating": "beoordeling",
"search_filters_sort_option_date": "datum",
"search_filters_sort_option_date": "Upload datum",
"search_filters_sort_option_views": "keren bekeken",
"search_filters_type_label": "Type inhoud",
"search_filters_duration_label": "duur",
"search_filters_features_label": "eigenschappen",
"search_filters_sort_label": "sorteren",
"search_filters_date_option_hour": "uur",
"search_filters_date_option_hour": "Laatste uur",
"search_filters_date_option_today": "vandaag",
"search_filters_date_option_week": "week",
"search_filters_date_option_month": "maand",
@ -357,7 +357,7 @@
"footer_original_source_code": "Originele bron-code",
"footer_modfied_source_code": "Gewijzigde bron-code",
"adminprefs_modified_source_code_url_label": "URL naar gewijzigde bron-code-opslagplaats",
"next_steps_error_message": "Daarna moet u proberen om: ",
"next_steps_error_message": "Waarna u zou kunnen proberen om: ",
"footer_source_code": "Bron-code",
"search_filters_duration_option_long": "Lang (> 20 minuten)",
"preferences_quality_option_dash": "DASH (adaptieve kwaliteit)",
@ -477,7 +477,7 @@
"Song: ": "Lied: ",
"generic_channels_count": "{{count}} kanaal",
"generic_channels_count_plural": "{{count}} kanalen",
"Popular enabled: ": "Populair geactiveerd: ",
"Popular enabled: ": "Populair ingeschakeld: ",
"channel_tab_playlists_label": "Afspeellijsten",
"generic_button_edit": "Bewerken",
"Music in this video": "Muziek in deze video",

View File

@ -508,7 +508,7 @@
"toggle_theme": "Trocar tema",
"Add to playlist": "Adicionar à lista de reprodução",
"Add to playlist: ": "Adicionar à lista de reprodução: ",
"Answer": "Resposta",
"Answer": "Responder",
"Search for videos": "Procurar vídeos",
"carousel_slide": "Diapositivo {{current}} de{{total}}",
"carousel_skip": "Ignorar carrossel",

View File

@ -509,6 +509,9 @@
"Add to playlist: ": "Добавить в плейлист: ",
"Answer": "Ответить",
"Search for videos": "Поиск видео",
"The Popular feed has been disabled by the administrator.": "Популярная лента была отключена администратором.",
"toggle_theme": "Переключатель тем"
"The Popular feed has been disabled by the administrator.": "Лента популярного была отключена администратором.",
"toggle_theme": "Переключатель тем",
"carousel_slide": "Пролистано {{current}} из {{total}}",
"carousel_skip": "Пропустить всё",
"carousel_go_to": "Перейти к странице `x`"
}

View File

@ -257,13 +257,13 @@
"Video mode": "Mënyrë video",
"channel_tab_videos_label": "Video",
"search_filters_sort_option_rating": "Vlerësim",
"search_filters_sort_option_date": "Datë Ngarkimi",
"search_filters_sort_option_date": "Datë ngarkimi",
"search_filters_sort_option_views": "Numër parjesh",
"search_filters_type_label": "Lloj",
"search_filters_duration_label": "Kohëzgjatje",
"search_filters_features_label": "Veçori",
"search_filters_sort_label": "Renditi Sipas",
"search_filters_date_option_hour": "Orën e Fundit",
"search_filters_date_option_hour": "Orën e fundit",
"search_filters_date_option_today": "Sot",
"search_filters_duration_option_long": "E gjatë (> 20 minuta)",
"search_filters_features_option_hd": "HD",
@ -435,7 +435,7 @@
"tokens_count_plural": "{{count}} tokenë",
"preferences_save_player_pos_label": "Mba mend pozicionin e luajtjes: ",
"Import Invidious data": "Importoni të dhëna JSON Invidious",
"Import YouTube subscriptions": "Importoni pajtime YouTube/OPML",
"Import YouTube subscriptions": "Importoni pajtime YouTube CSV ose OPML",
"Export data as JSON": "Eksportoji të dhënat Invidious si JSON",
"preferences_vr_mode_label": "Video me ndërveprim 360 gradë (lyp WebGL): ",
"Shared `x`": "Ndarë me të tjerë më `x`",
@ -484,5 +484,13 @@
"Import YouTube watch history (.json)": "Importo historik parjesh YouTube (.json)",
"preferences_local_label": "Video përmes ndërmjetësi: ",
"Fallback captions: ": "Titra nga halli: ",
"Erroneous challenge": "Zgjidhje e gabuar"
"Erroneous challenge": "Zgjidhje e gabuar",
"Add to playlist: ": "Shtoje te luajlistë: ",
"Add to playlist": "Shtoje te luajlistë",
"Answer": "Përgjigje",
"Search for videos": "Kërko për video",
"The Popular feed has been disabled by the administrator.": "Prurja Popullore është çaktivizuar nga përgjegjësi.",
"carousel_skip": "Anashkaloje Rrotullamen",
"carousel_slide": "Diapozitiv {{current}} nga {{total}}",
"carousel_go_to": "Kalo te diapozitivi `x`"
}

View File

@ -320,13 +320,13 @@
"channel_tab_community_label": "Gemenskap",
"search_filters_sort_option_relevance": "Relevans",
"search_filters_sort_option_rating": "Rankning",
"search_filters_sort_option_date": "Uppladdnings Datum",
"search_filters_sort_option_date": "Uppladdnings datum",
"search_filters_sort_option_views": "Visningar",
"search_filters_type_label": "Typ",
"search_filters_duration_label": "Varaktighet",
"search_filters_features_label": "Funktioner",
"search_filters_sort_label": "Sortera efter",
"search_filters_date_option_hour": "Senaste Timmen",
"search_filters_date_option_hour": "Senaste timmen",
"search_filters_date_option_today": "Idag",
"search_filters_date_option_week": "Denna vecka",
"search_filters_date_option_month": "Denna månad",

View File

@ -322,13 +322,13 @@
"channel_tab_community_label": "Topluluk",
"search_filters_sort_option_relevance": "İlgi",
"search_filters_sort_option_rating": "Değerlendirme",
"search_filters_sort_option_date": "Yükleme Tarihi",
"search_filters_sort_option_date": "Yükleme tarihi",
"search_filters_sort_option_views": "Görüntüleme Sayısı",
"search_filters_type_label": "Tür",
"search_filters_duration_label": "Süre",
"search_filters_features_label": "Özellikler",
"search_filters_sort_label": "Sıralama Ölçütü",
"search_filters_date_option_hour": "Son Saat",
"search_filters_date_option_hour": "Son saat",
"search_filters_date_option_today": "Bugün",
"search_filters_date_option_week": "Bu Hafta",
"search_filters_date_option_month": "Bu Ay",

View File

@ -455,7 +455,7 @@
"search_filters_date_option_week": "Цей тиждень",
"search_filters_type_label": "Тип",
"search_filters_type_option_channel": "Канал",
"search_message_use_another_instance": " Можете також <a href=\"`x`\">пошукати іншим сервером</a>.",
"search_message_use_another_instance": "Можете також <a href=\"`x`\">пошукати на іншому сервері</a>.",
"search_filters_title": "Фільтри",
"search_filters_date_option_hour": "Остання година",
"search_filters_date_option_month": "Цей місяць",
@ -472,7 +472,7 @@
"search_filters_features_option_three_sixty": "360°",
"search_filters_features_option_hdr": "HDR",
"search_filters_sort_label": "Спершу",
"search_filters_sort_option_date": "Нещодавні",
"search_filters_sort_option_date": "Дата вивантаження",
"search_filters_apply_button": "Застосувати фільтри",
"search_filters_features_option_vr180": "VR180",
"search_filters_features_option_purchased": "Придбано",

View File

@ -338,13 +338,13 @@
"channel_tab_community_label": "社群",
"search_filters_sort_option_relevance": "關聯",
"search_filters_sort_option_rating": "評分",
"search_filters_sort_option_date": "日期",
"search_filters_sort_option_date": "上傳日期",
"search_filters_sort_option_views": "檢視",
"search_filters_type_label": "內容類型",
"search_filters_duration_label": "時長",
"search_filters_features_label": "特色",
"search_filters_sort_label": "排序",
"search_filters_date_option_hour": "小時",
"search_filters_date_option_hour": "最後一小時",
"search_filters_date_option_today": "今天",
"search_filters_date_option_week": "週",
"search_filters_date_option_month": "月",

2
mocks

@ -1 +1 @@
Subproject commit 11ec372f72747c09d48ffef04843f72be67d5b54
Subproject commit b55d58dea94f7144ff0205857dfa70ec14eaa872

View File

@ -10,16 +10,20 @@ shards:
backtracer:
git: https://github.com/sija/backtracer.cr.git
version: 1.2.1
version: 1.2.2
db:
git: https://github.com/crystal-lang/crystal-db.git
version: 0.10.1
version: 0.13.1
exception_page:
git: https://github.com/crystal-loot/exception_page.git
version: 0.2.2
http_proxy:
git: https://github.com/mamantoha/http_proxy.git
version: 0.10.3
kemal:
git: https://github.com/kemalcr/kemal.git
version: 1.1.2
@ -30,7 +34,7 @@ shards:
pg:
git: https://github.com/will/crystal-pg.git
version: 0.24.0
version: 0.28.0
protodec:
git: https://github.com/iv-org/protodec.git
@ -42,9 +46,9 @@ shards:
spectator:
git: https://github.com/icy-arctic-fox/spectator.git
version: 0.10.4
version: 0.10.6
sqlite3:
git: https://github.com/crystal-lang/crystal-sqlite3.git
version: 0.18.0
version: 0.21.0

View File

@ -1,21 +1,20 @@
name: invidious
version: 0.20.1
version: 2.20241110.0-dev
authors:
- Omar Roth <omarroth@protonmail.com>
- Invidious team
- Invidious team <contact@invidious.io>
- Contributors!
targets:
invidious:
main: src/invidious.cr
description: |
Invidious is an alternative front-end to YouTube
dependencies:
pg:
github: will/crystal-pg
version: ~> 0.24.0
version: ~> 0.28.0
sqlite3:
github: crystal-lang/crystal-sqlite3
version: ~> 0.18.0
version: ~> 0.21.0
kemal:
github: kemalcr/kemal
version: ~> 1.1.2
@ -28,6 +27,9 @@ dependencies:
athena-negotiation:
github: athena-framework/negotiation
version: ~> 0.1.1
http_proxy:
github: mamantoha/http_proxy
version: ~> 0.10.3
development_dependencies:
spectator:
@ -37,6 +39,10 @@ development_dependencies:
github: crystal-ameba/ameba
version: ~> 1.6.1
crystal: ">= 1.0.0, < 2.0.0"
crystal: ">= 1.10.0, < 2.0.0"
license: AGPLv3
repository: https://github.com/iv-org/invidious
homepage: https://invidious.io
documentation: https://docs.invidious.io

View File

@ -27,8 +27,8 @@ Spectator.describe Invidious::Hashtag do
expect(video_11.length_seconds).to eq((56.minutes + 41.seconds).total_seconds.to_i32)
expect(video_11.views).to eq(40_504_893)
expect(video_11.live_now).to be_false
expect(video_11.premium).to be_false
expect(video_11.badges.live_now?).to be_false
expect(video_11.badges.premium?).to be_false
expect(video_11.premiere_timestamp).to be_nil
#
@ -49,8 +49,8 @@ Spectator.describe Invidious::Hashtag do
expect(video_35.length_seconds).to eq((3.minutes + 14.seconds).total_seconds.to_i32)
expect(video_35.views).to eq(30_790_049)
expect(video_35.live_now).to be_false
expect(video_35.premium).to be_false
expect(video_35.badges.live_now?).to be_false
expect(video_35.badges.premium?).to be_false
expect(video_35.premiere_timestamp).to be_nil
end
@ -80,8 +80,8 @@ Spectator.describe Invidious::Hashtag do
expect(video_41.length_seconds).to eq((1.hour).total_seconds.to_i32)
expect(video_41.views).to eq(63_240)
expect(video_41.live_now).to be_false
expect(video_41.premium).to be_false
expect(video_41.badges.live_now?).to be_false
expect(video_41.badges.premium?).to be_false
expect(video_41.premiere_timestamp).to be_nil
#
@ -102,8 +102,8 @@ Spectator.describe Invidious::Hashtag do
expect(video_48.length_seconds).to eq((35.minutes + 46.seconds).total_seconds.to_i32)
expect(video_48.views).to eq(68_704)
expect(video_48.live_now).to be_false
expect(video_48.premium).to be_false
expect(video_48.badges.live_now?).to be_false
expect(video_48.badges.premium?).to be_false
expect(video_48.premiere_timestamp).to be_nil
end
end

View File

@ -17,8 +17,8 @@ Spectator.describe "parse_video_info" do
# Basic video infos
expect(info["title"].as_s).to eq("I Gave My 100,000,000th Subscriber An Island")
expect(info["views"].as_i).to eq(126_573_823)
expect(info["likes"].as_i).to eq(5_157_654)
expect(info["views"].as_i).to eq(220_226_287)
expect(info["likes"].as_i).to eq(6_870_691)
# For some reason the video length from VideoDetails and the
# one from microformat differs by 1s...
@ -48,12 +48,12 @@ Spectator.describe "parse_video_info" do
expect(info["relatedVideos"].as_a.size).to eq(20)
expect(info["relatedVideos"][0]["id"]).to eq("Hwybp38GnZw")
expect(info["relatedVideos"][0]["title"]).to eq("I Built Willy Wonka's Chocolate Factory!")
expect(info["relatedVideos"][0]["id"]).to eq("krsBRQbOPQ4")
expect(info["relatedVideos"][0]["title"]).to eq("$1 vs $250,000,000 Private Island!")
expect(info["relatedVideos"][0]["author"]).to eq("MrBeast")
expect(info["relatedVideos"][0]["ucid"]).to eq("UCX6OQ3DkcsbYNE6H8uQQuVA")
expect(info["relatedVideos"][0]["view_count"]).to eq("179877630")
expect(info["relatedVideos"][0]["short_view_count"]).to eq("179M")
expect(info["relatedVideos"][0]["view_count"]).to eq("230617484")
expect(info["relatedVideos"][0]["short_view_count"]).to eq("230M")
expect(info["relatedVideos"][0]["author_verified"]).to eq("true")
# Description
@ -76,11 +76,11 @@ Spectator.describe "parse_video_info" do
expect(info["ucid"].as_s).to eq("UCX6OQ3DkcsbYNE6H8uQQuVA")
expect(info["authorThumbnail"].as_s).to eq(
"https://yt3.ggpht.com/ytc/AL5GRJVuqw82ERvHzsmBxL7avr1dpBtsVIXcEzBPZaloFg=s48-c-k-c0x00ffffff-no-rj"
"https://yt3.ggpht.com/fxGKYucJAVme-Yz4fsdCroCFCrANWqw0ql4GYuvx8Uq4l_euNJHgE-w9MTkLQA805vWCi-kE0g=s48-c-k-c0x00ffffff-no-rj"
)
expect(info["authorVerified"].as_bool).to be_true
expect(info["subCountText"].as_s).to eq("143M")
expect(info["subCountText"].as_s).to eq("320M")
end
it "parses a regular video with no descrition/comments" do
@ -99,8 +99,8 @@ Spectator.describe "parse_video_info" do
# Basic video infos
expect(info["title"].as_s).to eq("Chris Rea - Auberge")
expect(info["views"].as_i).to eq(10_943_126)
expect(info["likes"].as_i).to eq(0)
expect(info["views"].as_i).to eq(14_324_584)
expect(info["likes"].as_i).to eq(35_870)
expect(info["lengthSeconds"].as_i).to eq(283_i64)
expect(info["published"].as_s).to eq("2012-05-21T00:00:00Z")
@ -132,14 +132,14 @@ Spectator.describe "parse_video_info" do
# Related videos
expect(info["relatedVideos"].as_a.size).to eq(19)
expect(info["relatedVideos"].as_a.size).to eq(20)
expect(info["relatedVideos"][0]["id"]).to eq("Ww3KeZ2_Yv4")
expect(info["relatedVideos"][0]["title"]).to eq("Chris Rea")
expect(info["relatedVideos"][0]["author"]).to eq("PanMusic")
expect(info["relatedVideos"][0]["ucid"]).to eq("UCsKAPSuh1iNbLWUga_igPyA")
expect(info["relatedVideos"][0]["view_count"]).to eq("31581")
expect(info["relatedVideos"][0]["short_view_count"]).to eq("31K")
expect(info["relatedVideos"][0]["id"]).to eq("gUUdQfnshJ4")
expect(info["relatedVideos"][0]["title"]).to eq("Chris Rea - The Road To Hell 1989 Full Version")
expect(info["relatedVideos"][0]["author"]).to eq("NEA ZIXNH")
expect(info["relatedVideos"][0]["ucid"]).to eq("UCYMEOGcvav3gCgImK2J07CQ")
expect(info["relatedVideos"][0]["view_count"]).to eq("53298661")
expect(info["relatedVideos"][0]["short_view_count"]).to eq("53M")
expect(info["relatedVideos"][0]["author_verified"]).to eq("false")
# Description
@ -156,11 +156,13 @@ Spectator.describe "parse_video_info" do
# Author infos
expect(info["author"].as_s).to eq("ChrisReaOfficial")
expect(info["author"].as_s).to eq("ChrisReaVideos")
expect(info["ucid"].as_s).to eq("UC_5q6nWPbD30-y6oiWF_oNA")
expect(info["authorThumbnail"].as_s).to be_empty
expect(info["authorThumbnail"].as_s).to eq(
"https://yt3.ggpht.com/ytc/AIdro_n71nsegpKfjeRKwn1JJmK5IVMh_7j5m_h3_1KnUUg=s48-c-k-c0x00ffffff-no-rj"
)
expect(info["authorVerified"].as_bool).to be_false
expect(info["subCountText"].as_s).to eq("-")
expect(info["subCountText"].as_s).to eq("3.11K")
end
end

View File

@ -23,6 +23,7 @@ require "kilt"
require "./ext/kemal_content_for.cr"
require "./ext/kemal_static_file_handler.cr"
require "http_proxy"
require "athena-negotiation"
require "openssl/hmac"
require "option_parser"
@ -92,6 +93,10 @@ SOFTWARE = {
YT_POOL = YoutubeConnectionPool.new(YT_URL, capacity: CONFIG.pool_size)
# Image request pool
GGPHT_POOL = YoutubeConnectionPool.new(URI.parse("https://yt3.ggpht.com"), capacity: CONFIG.pool_size)
# CLI
Kemal.config.extra_options do |parser|
parser.banner = "Usage: invidious [arguments]"
@ -117,6 +122,9 @@ Kemal.config.extra_options do |parser|
parser.on("-l LEVEL", "--log-level=LEVEL", "Log level, one of #{LogLevel.values} (default: #{CONFIG.log_level})") do |log_level|
CONFIG.log_level = LogLevel.parse(log_level)
end
parser.on("-k", "--colorize", "Colorize logs") do
CONFIG.colorize_logs = true
end
parser.on("-v", "--version", "Print version") do
puts SOFTWARE.to_pretty_json
exit
@ -133,7 +141,7 @@ if CONFIG.output.upcase != "STDOUT"
FileUtils.mkdir_p(File.dirname(CONFIG.output))
end
OUTPUT = CONFIG.output.upcase == "STDOUT" ? STDOUT : File.open(CONFIG.output, mode: "a")
LOGGER = Invidious::LogHandler.new(OUTPUT, CONFIG.log_level)
LOGGER = Invidious::LogHandler.new(OUTPUT, CONFIG.log_level, CONFIG.colorize_logs)
# Check table integrity
Invidious::Database.check_integrity(CONFIG)
@ -189,6 +197,8 @@ Invidious::Jobs.register Invidious::Jobs::NotificationJob.new(CONNECTION_CHANNEL
Invidious::Jobs.register Invidious::Jobs::ClearExpiredItemsJob.new
Invidious::Jobs.register Invidious::Jobs::InstanceListRefreshJob.new
Invidious::Jobs.start_all
def popular_videos

View File

@ -223,7 +223,7 @@ def fetch_channel(ucid, pull_all_videos : Bool)
length_seconds = channel_video.try &.length_seconds
length_seconds ||= 0
live_now = channel_video.try &.live_now
live_now = channel_video.try &.badges.live_now?
live_now ||= false
premiere_timestamp = channel_video.try &.premiere_timestamp
@ -275,7 +275,7 @@ def fetch_channel(ucid, pull_all_videos : Bool)
ucid: video.ucid,
author: video.author,
length_seconds: video.length_seconds,
live_now: video.live_now,
live_now: video.badges.live_now?,
premiere_timestamp: video.premiere_timestamp,
views: video.views,
})

View File

@ -1,78 +1,3 @@
def produce_channel_content_continuation(ucid, content_type, page = 1, auto_generated = nil, sort_by = "newest", v2 = false)
object_inner_2 = {
"2:0:embedded" => {
"1:0:varint" => 0_i64,
},
"5:varint" => 50_i64,
"6:varint" => 1_i64,
"7:varint" => (page * 30).to_i64,
"9:varint" => 1_i64,
"10:varint" => 0_i64,
}
object_inner_2_encoded = object_inner_2
.try { |i| Protodec::Any.cast_json(i) }
.try { |i| Protodec::Any.from_json(i) }
.try { |i| Base64.urlsafe_encode(i) }
.try { |i| URI.encode_www_form(i) }
content_type_numerical =
case content_type
when "videos" then 15
when "livestreams" then 14
else 15 # Fallback to "videos"
end
sort_by_numerical =
case sort_by
when "newest" then 1_i64
when "popular" then 2_i64
when "oldest" then 4_i64
else 1_i64 # Fallback to "newest"
end
object_inner_1 = {
"110:embedded" => {
"3:embedded" => {
"#{content_type_numerical}:embedded" => {
"1:embedded" => {
"1:string" => object_inner_2_encoded,
},
"2:embedded" => {
"1:string" => "00000000-0000-0000-0000-000000000000",
},
"3:varint" => sort_by_numerical,
},
},
},
}
object_inner_1_encoded = object_inner_1
.try { |i| Protodec::Any.cast_json(i) }
.try { |i| Protodec::Any.from_json(i) }
.try { |i| Base64.urlsafe_encode(i) }
.try { |i| URI.encode_www_form(i) }
object = {
"80226972:embedded" => {
"2:string" => ucid,
"3:string" => object_inner_1_encoded,
"35:string" => "browse-feed#{ucid}videos102",
},
}
continuation = object.try { |i| Protodec::Any.cast_json(i) }
.try { |i| Protodec::Any.from_json(i) }
.try { |i| Base64.urlsafe_encode(i) }
.try { |i| URI.encode_www_form(i) }
return continuation
end
def make_initial_content_ctoken(ucid, content_type, sort_by) : String
return produce_channel_content_continuation(ucid, content_type, sort_by: sort_by)
end
module Invidious::Channel::Tabs
extend self
@ -101,7 +26,7 @@ module Invidious::Channel::Tabs
end
def get_videos(author : String, ucid : String, *, continuation : String? = nil, sort_by = "newest")
continuation ||= make_initial_content_ctoken(ucid, "videos", sort_by)
continuation ||= make_initial_videos_ctoken(ucid, sort_by)
initial_data = YoutubeAPI.browse(continuation: continuation)
return extract_items(initial_data, author, ucid)
@ -130,14 +55,10 @@ module Invidious::Channel::Tabs
# Shorts
# -------------------
def get_shorts(channel : AboutChannel, continuation : String? = nil)
if continuation.nil?
# EgZzaG9ydHPyBgUKA5oBAA%3D%3D is the protobuf object to load "shorts"
# TODO: try to extract the continuation tokens that allows other sorting options
initial_data = YoutubeAPI.browse(channel.ucid, params: "EgZzaG9ydHPyBgUKA5oBAA%3D%3D")
else
def get_shorts(channel : AboutChannel, *, continuation : String? = nil, sort_by = "newest")
continuation ||= make_initial_shorts_ctoken(channel.ucid, sort_by)
initial_data = YoutubeAPI.browse(continuation: continuation)
end
return extract_items(initial_data, channel.author, channel.ucid)
end
@ -145,9 +66,8 @@ module Invidious::Channel::Tabs
# Livestreams
# -------------------
def get_livestreams(channel : AboutChannel, continuation : String? = nil, sort_by = "newest")
continuation ||= make_initial_content_ctoken(channel.ucid, "livestreams", sort_by)
def get_livestreams(channel : AboutChannel, *, continuation : String? = nil, sort_by = "newest")
continuation ||= make_initial_livestreams_ctoken(channel.ucid, sort_by)
initial_data = YoutubeAPI.browse(continuation: continuation)
return extract_items(initial_data, channel.author, channel.ucid)
@ -171,4 +91,102 @@ module Invidious::Channel::Tabs
return items, next_continuation
end
# -------------------
# C-tokens
# -------------------
private def sort_options_videos_short(sort_by : String)
case sort_by
when "newest" then return 4_i64
when "popular" then return 2_i64
when "oldest" then return 5_i64
else return 4_i64 # Fallback to "newest"
end
end
# Generate the initial "continuation token" to get the first page of the
# "videos" tab. The following page requires the ctoken provided in that
# first page, and so on.
private def make_initial_videos_ctoken(ucid : String, sort_by = "newest")
object = {
"15:embedded" => {
"2:embedded" => {
"1:string" => "00000000-0000-0000-0000-000000000000",
},
"4:varint" => sort_options_videos_short(sort_by),
},
}
return channel_ctoken_wrap(ucid, object)
end
# Generate the initial "continuation token" to get the first page of the
# "shorts" tab. The following page requires the ctoken provided in that
# first page, and so on.
private def make_initial_shorts_ctoken(ucid : String, sort_by = "newest")
object = {
"10:embedded" => {
"2:embedded" => {
"1:string" => "00000000-0000-0000-0000-000000000000",
},
"4:varint" => sort_options_videos_short(sort_by),
},
}
return channel_ctoken_wrap(ucid, object)
end
# Generate the initial "continuation token" to get the first page of the
# "livestreams" tab. The following page requires the ctoken provided in that
# first page, and so on.
private def make_initial_livestreams_ctoken(ucid : String, sort_by = "newest")
sort_by_numerical =
case sort_by
when "newest" then 12_i64
when "popular" then 14_i64
when "oldest" then 13_i64
else 12_i64 # Fallback to "newest"
end
object = {
"14:embedded" => {
"2:embedded" => {
"1:string" => "00000000-0000-0000-0000-000000000000",
},
"5:varint" => sort_by_numerical,
},
}
return channel_ctoken_wrap(ucid, object)
end
# The protobuf structure common between videos/shorts/livestreams
private def channel_ctoken_wrap(ucid : String, object)
object_inner = {
"110:embedded" => {
"3:embedded" => object,
},
}
object_inner_encoded = object_inner
.try { |i| Protodec::Any.cast_json(i) }
.try { |i| Protodec::Any.from_json(i) }
.try { |i| Base64.urlsafe_encode(i) }
.try { |i| URI.encode_www_form(i) }
object = {
"80226972:embedded" => {
"2:string" => ucid,
"3:string" => object_inner_encoded,
},
}
continuation = object.try { |i| Protodec::Any.cast_json(i) }
.try { |i| Protodec::Any.from_json(i) }
.try { |i| Base64.urlsafe_encode(i) }
.try { |i| URI.encode_www_form(i) }
return continuation
end
end

View File

@ -13,6 +13,7 @@ struct ConfigPreferences
property annotations : Bool = false
property annotations_subscribed : Bool = false
property preload : Bool = true
property autoplay : Bool = false
property captions : Array(String) = ["", "", ""]
property comments : Array(String) = ["youtube", ""]
@ -54,6 +55,15 @@ struct ConfigPreferences
end
end
struct HTTPProxyConfig
include YAML::Serializable
property user : String
property password : String
property host : String
property port : Int32
end
class Config
include YAML::Serializable
@ -68,6 +78,8 @@ class Config
property output : String = "STDOUT"
# Default log level, valid YAML values are ints and strings, see src/invidious/helpers/logger.cr
property log_level : LogLevel = LogLevel::Info
# Enables colors in logs. Useful for debugging purposes
property colorize_logs : Bool = false
# Database configuration with separate parameters (username, hostname, etc)
property db : DBConfig? = nil
@ -128,6 +140,8 @@ class Config
property host_binding : String = "0.0.0.0"
# Pool size for HTTP requests to youtube.com and ytimg.com (each domain has a separate pool of `pool_size`)
property pool_size : Int32 = 100
# HTTP Proxy configuration
property http_proxy : HTTPProxyConfig? = nil
# Use Innertube's transcripts API instead of timedtext for closed captions
property use_innertube_for_captions : Bool = false

View File

@ -18,6 +18,40 @@ end
class HTTP::Client
property family : Socket::Family = Socket::Family::UNSPEC
# Override stdlib to automatically initialize proxy if configured
#
# Accurate as of crystal 1.12.1
def initialize(@host : String, port = nil, tls : TLSContext = nil)
check_host_only(@host)
{% if flag?(:without_openssl) %}
if tls
raise "HTTP::Client TLS is disabled because `-D without_openssl` was passed at compile time"
end
@tls = nil
{% else %}
@tls = case tls
when true
OpenSSL::SSL::Context::Client.new
when OpenSSL::SSL::Context::Client
tls
when false, nil
nil
end
{% end %}
@port = (port || (@tls ? 443 : 80)).to_i
self.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
end
def initialize(@io : IO, @host = "", @port = 80)
@reconnect = false
self.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
end
private def io
io = @io
return io if io

View File

@ -43,6 +43,8 @@ def error_template_helper(env : HTTP::Server::Context, status_code : Int32, exce
# URLs for the error message below
url_faq = "https://github.com/iv-org/documentation/blob/master/docs/faq.md"
url_search_issues = "https://github.com/iv-org/invidious/issues"
url_search_issues += "?q=is:issue+is:open+"
url_search_issues += URI.encode_www_form("[Bug] #{issue_title}")
url_switch = "https://redirect.invidious.io" + env.request.resource

View File

@ -1,8 +1,22 @@
# Languages requiring a better level of translation (at least 20%)
# to be added to the list below:
#
# "af" => "", # Afrikaans
# "az" => "", # Azerbaijani
# "be" => "", # Belarusian
# "bn_BD" => "", # Bengali (Bangladesh)
# "ia" => "", # Interlingua
# "or" => "", # Odia
# "tk" => "", # Turkmen
# "tok => "", # Toki Pona
#
LOCALES_LIST = {
"ar" => "العربية", # Arabic
"bg" => "български", # Bulgarian
"bn" => "বাংলা", # Bengali
"ca" => "Català", # Catalan
"cs" => "Čeština", # Czech
"cy" => "Cymraeg", # Welsh
"da" => "Dansk", # Danish
"de" => "Deutsch", # German
"el" => "Ελληνικά", # Greek
@ -23,6 +37,7 @@ LOCALES_LIST = {
"it" => "Italiano", # Italian
"ja" => "日本語", # Japanese
"ko" => "한국어", # Korean
"lmo" => "Lombard", # Lombard
"lt" => "Lietuvių", # Lithuanian
"nb-NO" => "Norsk bokmål", # Norwegian Bokmål
"nl" => "Nederlands", # Dutch

View File

@ -1,3 +1,5 @@
require "colorize"
enum LogLevel
All = 0
Trace = 1
@ -10,7 +12,9 @@ enum LogLevel
end
class Invidious::LogHandler < Kemal::BaseLogHandler
def initialize(@io : IO = STDOUT, @level = LogLevel::Debug)
def initialize(@io : IO = STDOUT, @level = LogLevel::Debug, use_color : Bool = true)
Colorize.enabled = use_color
Colorize.on_tty_only!
end
def call(context : HTTP::Server::Context)
@ -39,10 +43,22 @@ class Invidious::LogHandler < Kemal::BaseLogHandler
@io.flush
end
def color(level)
case level
when LogLevel::Trace then :cyan
when LogLevel::Debug then :green
when LogLevel::Info then :white
when LogLevel::Warn then :yellow
when LogLevel::Error then :red
when LogLevel::Fatal then :magenta
else :default
end
end
{% for level in %w(trace debug info warn error fatal) %}
def {{level.id}}(message : String)
if LogLevel::{{level.id.capitalize}} >= @level
puts("#{Time.utc} [{{level.id}}] #{message}")
puts("#{Time.utc} [{{level.id}}] #{message}".colorize(color(LogLevel::{{level.id.capitalize}})))
end
end
{% end %}

View File

@ -1,3 +1,16 @@
@[Flags]
enum VideoBadges
LiveNow
Premium
ThreeD
FourK
New
EightK
VR180
VR360
ClosedCaptions
end
struct SearchVideo
include DB::Serializable
@ -9,10 +22,9 @@ struct SearchVideo
property views : Int64
property description_html : String
property length_seconds : Int32
property live_now : Bool
property premium : Bool
property premiere_timestamp : Time?
property author_verified : Bool
property badges : VideoBadges
def to_xml(auto_generated, query_params, xml : XML::Builder)
query_params["v"] = self.id
@ -88,13 +100,20 @@ struct SearchVideo
json.field "published", self.published.to_unix
json.field "publishedText", translate(locale, "`x` ago", recode_date(self.published, locale))
json.field "lengthSeconds", self.length_seconds
json.field "liveNow", self.live_now
json.field "premium", self.premium
json.field "liveNow", self.badges.live_now?
json.field "premium", self.badges.premium?
json.field "isUpcoming", self.upcoming?
if self.premiere_timestamp
json.field "premiereTimestamp", self.premiere_timestamp.try &.to_unix
end
json.field "isNew", self.badges.new?
json.field "is4k", self.badges.four_k?
json.field "is8k", self.badges.eight_k?
json.field "isVr180", self.badges.vr180?
json.field "isVr360", self.badges.vr360?
json.field "is3d", self.badges.three_d?
json.field "hasCaptions", self.badges.closed_captions?
end
end

View File

@ -175,8 +175,9 @@ module Invidious::SigHelper
@queue = {} of TransactionID => Transaction
@conn : Connection
@uri_or_path : String
def initialize(uri_or_path)
def initialize(@uri_or_path)
@conn = Connection.new(uri_or_path)
listen
end
@ -186,10 +187,26 @@ module Invidious::SigHelper
LOGGER.debug("SigHelper: Multiplexor listening")
# TODO: reopen socket if unexpectedly closed
spawn do
loop do
begin
receive_data
rescue ex
LOGGER.info("SigHelper: Connection to helper died with '#{ex.message}' trying to reconnect...")
# We close the socket because for some reason is not closed.
@conn.close
loop do
begin
@conn = Connection.new(@uri_or_path)
LOGGER.info("SigHelper: Reconnected to SigHelper!")
rescue ex
LOGGER.debug("SigHelper: Reconnection to helper unsuccessful with error '#{ex.message}'. Retrying")
sleep 500.milliseconds
next
end
break if !@conn.closed?
end
end
Fiber.yield
end
end

View File

@ -323,68 +323,6 @@ def parse_range(range)
return 0_i64, nil
end
def fetch_random_instance
begin
instance_api_client = make_client(URI.parse("https://api.invidious.io"))
# Timeouts
instance_api_client.connect_timeout = 10.seconds
instance_api_client.dns_timeout = 10.seconds
instance_list = JSON.parse(instance_api_client.get("/instances.json").body).as_a
instance_api_client.close
rescue Socket::ConnectError | IO::TimeoutError | JSON::ParseException
instance_list = [] of JSON::Any
end
filtered_instance_list = [] of String
instance_list.each do |data|
# TODO Check if current URL is onion instance and use .onion types if so.
if data[1]["type"] == "https"
# Instances can have statistics disabled, which is an requirement of version validation.
# as_nil? doesn't exist. Thus we'll have to handle the error raised if as_nil fails.
begin
data[1]["stats"].as_nil
next
rescue TypeCastError
end
# stats endpoint could also lack the software dict.
next if data[1]["stats"]["software"]?.nil?
# Makes sure the instance isn't too outdated.
if remote_version = data[1]["stats"]?.try &.["software"]?.try &.["version"]
remote_commit_date = remote_version.as_s.match(/\d{4}\.\d{2}\.\d{2}/)
next if !remote_commit_date
remote_commit_date = Time.parse(remote_commit_date[0], "%Y.%m.%d", Time::Location::UTC)
local_commit_date = Time.parse(CURRENT_VERSION, "%Y.%m.%d", Time::Location::UTC)
next if (remote_commit_date - local_commit_date).abs.days > 30
begin
data[1]["monitor"].as_nil
health = data[1]["monitor"].as_h["dailyRatios"][0].as_h["ratio"]
filtered_instance_list << data[0].as_s if health.to_s.to_f > 90
rescue TypeCastError
# We can't check the health if the monitoring is broken. Thus we'll just add it to the list
# and move on. Ideally we'll ignore any instance that has broken health monitoring but due to the fact that
# it's an error that often occurs with all the instances at the same time, we have to just skip the check.
filtered_instance_list << data[0].as_s
end
end
end
end
# If for some reason no instances managed to get fetched successfully then we'll just redirect to redirect.invidious.io
if filtered_instance_list.size == 0
return "redirect.invidious.io"
end
return filtered_instance_list.sample(1)[0]
end
def reduce_uri(uri : URI | String, max_length : Int32 = 50, suffix : String = "") : String
str = uri.to_s.sub(/^https?:\/\//, "")
if str.size > max_length

View File

@ -0,0 +1,97 @@
class Invidious::Jobs::InstanceListRefreshJob < Invidious::Jobs::BaseJob
# We update the internals of a constant as so it can be accessed from anywhere
# within the codebase
#
# "INSTANCES" => Array(Tuple(String, String)) # region, instance
INSTANCES = {"INSTANCES" => [] of Tuple(String, String)}
def initialize
end
def begin
loop do
refresh_instances
LOGGER.info("InstanceListRefreshJob: Done, sleeping for 30 minutes")
sleep 30.minute
Fiber.yield
end
end
# Refreshes the list of instances used for redirects.
#
# Does the following three checks for each instance
# - Is it a clear-net instance?
# - Is it an instance with a good uptime?
# - Is it an updated instance?
private def refresh_instances
raw_instance_list = self.fetch_instances
filtered_instance_list = [] of Tuple(String, String)
raw_instance_list.each do |instance_data|
# TODO allow Tor hidden service instances when the current instance
# is also a hidden service. Same for i2p and any other non-clearnet instances.
begin
domain = instance_data[0]
info = instance_data[1]
stats = info["stats"]
next unless info["type"] == "https"
next if bad_uptime?(info["monitor"])
next if outdated?(stats["software"]["version"])
filtered_instance_list << {info["region"].as_s, domain.as_s}
rescue ex
if domain
LOGGER.info("InstanceListRefreshJob: failed to parse information from '#{domain}' because \"#{ex}\"\n\"#{ex.backtrace.join('\n')}\" ")
else
LOGGER.info("InstanceListRefreshJob: failed to parse information from an instance because \"#{ex}\"\n\"#{ex.backtrace.join('\n')}\" ")
end
end
end
if !filtered_instance_list.empty?
INSTANCES["INSTANCES"] = filtered_instance_list
end
end
# Fetches information regarding instances from api.invidious.io or an otherwise configured URL
private def fetch_instances : Array(JSON::Any)
begin
# We directly call the stdlib HTTP::Client here as it allows us to negate the effects
# of the force_resolve config option. This is needed as api.invidious.io does not support ipv6
# and as such the following request raises if we were to use force_resolve with the ipv6 value.
instance_api_client = HTTP::Client.new(URI.parse("https://api.invidious.io"))
# Timeouts
instance_api_client.connect_timeout = 10.seconds
instance_api_client.dns_timeout = 10.seconds
raw_instance_list = JSON.parse(instance_api_client.get("/instances.json").body).as_a
instance_api_client.close
rescue ex : Socket::ConnectError | IO::TimeoutError | JSON::ParseException
raw_instance_list = [] of JSON::Any
end
return raw_instance_list
end
# Checks if the given target instance is outdated
private def outdated?(target_instance_version) : Bool
remote_commit_date = target_instance_version.as_s.match(/\d{4}\.\d{2}\.\d{2}/)
return false if !remote_commit_date
remote_commit_date = Time.parse(remote_commit_date[0], "%Y.%m.%d", Time::Location::UTC)
local_commit_date = Time.parse(CURRENT_VERSION, "%Y.%m.%d", Time::Location::UTC)
return (remote_commit_date - local_commit_date).abs.days > 30
end
# Checks if the uptime of the target instance is greater than 90% over a 30 day period
private def bad_uptime?(target_instance_health_monitor) : Bool
return true if !target_instance_health_monitor["down"].as_bool == false
return true if target_instance_health_monitor["uptime"].as_f < 90
return false
end
end

View File

@ -270,7 +270,7 @@ end
def subscribe_playlist(user, playlist)
playlist = InvidiousPlaylist.new({
title: playlist.title.byte_slice(0, 150),
title: playlist.title[..150],
id: playlist.id,
author: user.email,
description: "", # Max 5000 characters

View File

@ -197,6 +197,7 @@ module Invidious::Routes::API::V1::Channels
get_channel()
# Retrieve continuation from URL parameters
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
continuation = env.params.query["continuation"]?
if channel.is_age_gated
@ -211,7 +212,7 @@ module Invidious::Routes::API::V1::Channels
else
begin
videos, next_continuation = Channel::Tabs.get_shorts(
channel, continuation: continuation
channel, continuation: continuation, sort_by: sort_by
)
rescue ex
return error_json(500, ex)

View File

@ -36,9 +36,7 @@ module Invidious::Routes::API::V1::Search
query = env.params.query["q"]? || ""
begin
client = HTTP::Client.new("suggestqueries-clients6.youtube.com")
client.before_request { |r| add_yt_headers(r) }
client = make_client(URI.parse("https://suggestqueries-clients6.youtube.com"), force_youtube_headers: true)
url = "/complete/search?client=youtube&hl=en&gl=#{region}&q=#{URI.encode_www_form(query)}&gs_ri=youtube&ds=yt"
response = client.get(url).body

View File

@ -20,10 +20,11 @@ module Invidious::Routes::Channels
sort_by = env.params.query["sort_by"]?.try &.downcase
if channel.auto_generated
sort_by ||= "last"
sort_options = {"last", "oldest", "newest"}
items, next_continuation = fetch_channel_playlists(
channel.ucid, channel.author, continuation, (sort_by || "last")
channel.ucid, channel.author, continuation, sort_by
)
items.uniq! do |item|
@ -49,9 +50,11 @@ module Invidious::Routes::Channels
end
next_continuation = nil
else
sort_by ||= "newest"
sort_options = {"newest", "oldest", "popular"}
items, next_continuation = Channel::Tabs.get_videos(
channel, continuation: continuation, sort_by: (sort_by || "newest")
items, next_continuation = Channel::Tabs.get_60_videos(
channel, continuation: continuation, sort_by: sort_by
)
end
end
@ -82,13 +85,12 @@ module Invidious::Routes::Channels
end
next_continuation = nil
else
# TODO: support sort option for shorts
sort_by = ""
sort_options = [] of String
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
sort_options = {"newest", "oldest", "popular"}
# Fetch items and continuation token
items, next_continuation = Channel::Tabs.get_shorts(
channel, continuation: continuation
channel, continuation: continuation, sort_by: sort_by
)
end

View File

@ -197,11 +197,9 @@ module Invidious::Routes::Feeds
views: views,
description_html: description_html,
length_seconds: 0,
live_now: false,
paid: false,
premium: false,
premiere_timestamp: nil,
author_verified: false,
badges: VideoBadges::None,
})
end

View File

@ -11,29 +11,9 @@ module Invidious::Routes::Images
end
end
# We're encapsulating this into a proc in order to easily reuse this
# portion of the code for each request block below.
request_proc = ->(response : HTTP::Client::Response) {
env.response.status_code = response.status_code
response.headers.each do |key, value|
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
env.response.headers[key] = value
end
end
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300
env.response.headers.delete("Transfer-Encoding")
return
end
proxy_file(response, env)
}
begin
HTTP::Client.get("https://yt3.ggpht.com#{url}") do |resp|
return request_proc.call(resp)
GGPHT_POOL.client &.get(url, headers) do |resp|
return self.proxy_image(env, resp)
end
rescue ex
end
@ -61,27 +41,10 @@ module Invidious::Routes::Images
end
end
request_proc = ->(response : HTTP::Client::Response) {
env.response.status_code = response.status_code
response.headers.each do |key, value|
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
env.response.headers[key] = value
end
end
env.response.headers["Connection"] = "close"
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300
return env.response.headers.delete("Transfer-Encoding")
end
proxy_file(response, env)
}
begin
HTTP::Client.get("https://#{authority}.ytimg.com#{url}") do |resp|
return request_proc.call(resp)
get_ytimg_pool(authority).client &.get(url, headers) do |resp|
env.response.headers["Connection"] = "close"
return self.proxy_image(env, resp)
end
rescue ex
end
@ -101,26 +64,9 @@ module Invidious::Routes::Images
end
end
request_proc = ->(response : HTTP::Client::Response) {
env.response.status_code = response.status_code
response.headers.each do |key, value|
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
env.response.headers[key] = value
end
end
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300 && response.status_code != 404
return env.response.headers.delete("Transfer-Encoding")
end
proxy_file(response, env)
}
begin
HTTP::Client.get("https://i9.ytimg.com#{url}") do |resp|
return request_proc.call(resp)
get_ytimg_pool("i9").client &.get(url, headers) do |resp|
return self.proxy_image(env, resp)
end
rescue ex
end
@ -165,8 +111,7 @@ module Invidious::Routes::Images
if name == "maxres.jpg"
build_thumbnails(id).each do |thumb|
thumbnail_resource_path = "/vi/#{id}/#{thumb[:url]}.jpg"
# This can likely be optimized into a (small) pool sometime in the future.
if HTTP::Client.head("https://i.ytimg.com#{thumbnail_resource_path}").status_code == 200
if get_ytimg_pool("i9").client &.head(thumbnail_resource_path, headers).status_code == 200
name = thumb[:url] + ".jpg"
break
end
@ -181,7 +126,15 @@ module Invidious::Routes::Images
end
end
request_proc = ->(response : HTTP::Client::Response) {
begin
get_ytimg_pool("i").client &.get(url, headers) do |resp|
return self.proxy_image(env, resp)
end
rescue ex
end
end
private def self.proxy_image(env, response)
env.response.status_code = response.status_code
response.headers.each do |key, value|
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
@ -191,19 +144,10 @@ module Invidious::Routes::Images
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300 && response.status_code != 404
if response.status_code >= 300
return env.response.headers.delete("Transfer-Encoding")
end
proxy_file(response, env)
}
begin
# This can likely be optimized into a (small) pool sometime in the future.
HTTP::Client.get("https://i.ytimg.com#{url}") do |resp|
return request_proc.call(resp)
end
rescue ex
end
return proxy_file(response, env)
end
end

View File

@ -40,7 +40,16 @@ module Invidious::Routes::Misc
def self.cross_instance_redirect(env)
referer = get_referer(env)
instance_url = fetch_random_instance
instance_list = Invidious::Jobs::InstanceListRefreshJob::INSTANCES["INSTANCES"]
if instance_list.empty?
instance_url = "redirect.invidious.io"
else
# Sample returns an array
# Instances are packaged as {region, domain} in the instance list
instance_url = instance_list.sample(1)[0][1]
end
env.redirect "https://#{instance_url}#{referer}"
end
end

View File

@ -27,6 +27,10 @@ module Invidious::Routes::PreferencesRoute
annotations_subscribed ||= "off"
annotations_subscribed = annotations_subscribed == "on"
preload = env.params.body["preload"]?.try &.as(String)
preload ||= "off"
preload = preload == "on"
autoplay = env.params.body["autoplay"]?.try &.as(String)
autoplay ||= "off"
autoplay = autoplay == "on"
@ -144,6 +148,7 @@ module Invidious::Routes::PreferencesRoute
preferences = Preferences.from_json({
annotations: annotations,
annotations_subscribed: annotations_subscribed,
preload: preload,
autoplay: autoplay,
captions: captions,
comments: comments,

View File

@ -42,7 +42,7 @@ module Invidious::Routes::VideoPlayback
headers["Range"] = "bytes=#{range_for_head}"
end
client = make_client(URI.parse(host), region, force_resolve = true)
client = make_client(URI.parse(host), region, force_resolve: true)
response = HTTP::Client::Response.new(500)
error = ""
5.times do
@ -57,7 +57,7 @@ module Invidious::Routes::VideoPlayback
if new_host != host
host = new_host
client.close
client = make_client(URI.parse(new_host), region, force_resolve = true)
client = make_client(URI.parse(new_host), region, force_resolve: true)
end
url = "#{location.request_target}&host=#{location.host}#{region ? "&region=#{region}" : ""}"
@ -71,7 +71,7 @@ module Invidious::Routes::VideoPlayback
fvip = "3"
host = "https://r#{fvip}---#{mn}.googlevideo.com"
client = make_client(URI.parse(host), region, force_resolve = true)
client = make_client(URI.parse(host), region, force_resolve: true)
rescue ex
error = ex.message
end
@ -196,7 +196,7 @@ module Invidious::Routes::VideoPlayback
break
else
client.close
client = make_client(URI.parse(host), region, force_resolve = true)
client = make_client(URI.parse(host), region, force_resolve: true)
end
end

View File

@ -243,17 +243,16 @@ module Invidious::Routing
# Channels
get "/api/v1/channels/:ucid", {{namespace}}::Channels, :home
get "/api/v1/channels/:ucid/latest", {{namespace}}::Channels, :latest
get "/api/v1/channels/:ucid/videos", {{namespace}}::Channels, :videos
get "/api/v1/channels/:ucid/shorts", {{namespace}}::Channels, :shorts
get "/api/v1/channels/:ucid/streams", {{namespace}}::Channels, :streams
get "/api/v1/channels/:ucid/podcasts", {{namespace}}::Channels, :podcasts
get "/api/v1/channels/:ucid/releases", {{namespace}}::Channels, :releases
get "/api/v1/channels/:ucid/playlists", {{namespace}}::Channels, :playlists
get "/api/v1/channels/:ucid/community", {{namespace}}::Channels, :community
get "/api/v1/channels/:ucid/channels", {{namespace}}::Channels, :channels
{% for route in {"videos", "latest", "playlists", "community", "search"} %}
get "/api/v1/channels/#{{{route}}}/:ucid", {{namespace}}::Channels, :{{route}}
get "/api/v1/channels/:ucid/#{{{route}}}", {{namespace}}::Channels, :{{route}}
{% end %}
get "/api/v1/channels/:ucid/search", {{namespace}}::Channels, :search
# Posts
get "/api/v1/post/:id", {{namespace}}::Channels, :post
@ -271,11 +270,6 @@ module Invidious::Routing
# Authenticated
# The notification APIs cannot be extracted yet! They require the *local* notifications constant defined in invidious.cr
#
# Invidious::Routing.get "/api/v1/auth/notifications", {{namespace}}::Authenticated, :notifications
# Invidious::Routing.post "/api/v1/auth/notifications", {{namespace}}::Authenticated, :notifications
get "/api/v1/auth/preferences", {{namespace}}::Authenticated, :get_preferences
post "/api/v1/auth/preferences", {{namespace}}::Authenticated, :set_preferences

View File

@ -4,6 +4,7 @@ struct Preferences
property annotations : Bool = CONFIG.default_user_preferences.annotations
property annotations_subscribed : Bool = CONFIG.default_user_preferences.annotations_subscribed
property preload : Bool = CONFIG.default_user_preferences.preload
property autoplay : Bool = CONFIG.default_user_preferences.autoplay
property automatic_instance_redirect : Bool = CONFIG.default_user_preferences.automatic_instance_redirect

View File

@ -26,12 +26,6 @@ struct Video
@[DB::Field(ignore: true)]
@captions = [] of Invidious::Videos::Captions::Metadata
@[DB::Field(ignore: true)]
property adaptive_fmts : Array(Hash(String, JSON::Any))?
@[DB::Field(ignore: true)]
property fmt_stream : Array(Hash(String, JSON::Any))?
@[DB::Field(ignore: true)]
property description : String?
@ -98,72 +92,24 @@ struct Video
# Methods for parsing streaming data
def convert_url(fmt)
if cfr = fmt["signatureCipher"]?.try { |json| HTTP::Params.parse(json.as_s) }
sp = cfr["sp"]
url = URI.parse(cfr["url"])
params = url.query_params
LOGGER.debug("Videos: Decoding '#{cfr}'")
unsig = DECRYPT_FUNCTION.try &.decrypt_signature(cfr["s"])
params[sp] = unsig if unsig
def fmt_stream : Array(Hash(String, JSON::Any))
if formats = info.dig?("streamingData", "formats")
return formats
.as_a.map(&.as_h)
.sort_by! { |f| f["width"]?.try &.as_i || 0 }
else
url = URI.parse(fmt["url"].as_s)
params = url.query_params
return [] of Hash(String, JSON::Any)
end
end
n = DECRYPT_FUNCTION.try &.decrypt_nsig(params["n"])
params["n"] = n if n
if token = CONFIG.po_token
params["pot"] = token
def adaptive_fmts : Array(Hash(String, JSON::Any))
if formats = info.dig?("streamingData", "adaptiveFormats")
return formats
.as_a.map(&.as_h)
.sort_by! { |f| f["width"]?.try &.as_i || 0 }
else
return [] of Hash(String, JSON::Any)
end
params["host"] = url.host.not_nil!
if region = self.info["region"]?.try &.as_s
params["region"] = region
end
url.query_params = params
LOGGER.trace("Videos: new url is '#{url}'")
return url.to_s
rescue ex
LOGGER.debug("Videos: Error when parsing video URL")
LOGGER.trace(ex.inspect_with_backtrace)
return ""
end
def fmt_stream
return @fmt_stream.as(Array(Hash(String, JSON::Any))) if @fmt_stream
fmt_stream = info.dig?("streamingData", "formats")
.try &.as_a.map &.as_h || [] of Hash(String, JSON::Any)
fmt_stream.each do |fmt|
fmt["url"] = JSON::Any.new(self.convert_url(fmt))
end
fmt_stream.sort_by! { |f| f["width"]?.try &.as_i || 0 }
@fmt_stream = fmt_stream
return @fmt_stream.as(Array(Hash(String, JSON::Any)))
end
def adaptive_fmts
return @adaptive_fmts.as(Array(Hash(String, JSON::Any))) if @adaptive_fmts
fmt_stream = info.dig("streamingData", "adaptiveFormats")
.try &.as_a.map &.as_h || [] of Hash(String, JSON::Any)
fmt_stream.each do |fmt|
fmt["url"] = JSON::Any.new(self.convert_url(fmt))
end
fmt_stream.sort_by! { |f| f["width"]?.try &.as_i || 0 }
@adaptive_fmts = fmt_stream
return @adaptive_fmts.as(Array(Hash(String, JSON::Any)))
end
def video_streams

View File

@ -123,6 +123,7 @@ module Invidious::Videos
"Esperanto",
"Estonian",
"Filipino",
"Filipino (auto-generated)",
"Finnish",
"French",
"French (auto-generated)",

View File

@ -102,8 +102,8 @@ def extract_video_info(video_id : String)
new_player_response = nil
# Don't use Android client if po_token is passed because po_token doesn't
# work for Android client.
# Don't use Android test suite client if po_token is passed because po_token doesn't
# work for Android test suite client.
if reason.nil? && CONFIG.po_token.nil?
# Fetch the video streams using an Android client in order to get the
# decrypted URLs and maybe fix throttling issues (#2194). See the
@ -113,15 +113,6 @@ def extract_video_info(video_id : String)
new_player_response = try_fetch_streaming_data(video_id, client_config)
end
# Last hope
# Only trigger if reason found and po_token or didn't work wth Android client.
# TvHtml5ScreenEmbed now requires sig helper for it to work but po_token is not required
# if the IP address is not blocked.
if CONFIG.po_token && reason || CONFIG.po_token.nil? && new_player_response.nil?
client_config.client_type = YoutubeAPI::ClientType::TvHtml5ScreenEmbed
new_player_response = try_fetch_streaming_data(video_id, client_config)
end
# Replace player response and reset reason
if !new_player_response.nil?
# Preserve captions & storyboard data before replacement
@ -132,10 +123,21 @@ def extract_video_info(video_id : String)
params.delete("reason")
end
{"captions", "playabilityStatus", "playerConfig", "storyboards", "streamingData"}.each do |f|
{"captions", "playabilityStatus", "playerConfig", "storyboards"}.each do |f|
params[f] = player_response[f] if player_response[f]?
end
# Convert URLs, if those are present
if streaming_data = player_response["streamingData"]?
%w[formats adaptiveFormats].each do |key|
streaming_data.as_h[key]?.try &.as_a.each do |format|
format.as_h["url"] = JSON::Any.new(convert_url(format))
end
end
params["streamingData"] = streaming_data
end
# Data structure version, for cache control
params["version"] = JSON::Any.new(Video::SCHEMA_VERSION.to_i64)
@ -185,10 +187,11 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
end
video_details = player_response.dig?("videoDetails")
microformat = player_response.dig?("microformat", "playerMicroformatRenderer")
if !(microformat = player_response.dig?("microformat", "playerMicroformatRenderer"))
microformat = {} of String => JSON::Any
end
raise BrokenTubeException.new("videoDetails") if !video_details
raise BrokenTubeException.new("microformat") if !microformat
# Basic video infos
@ -213,8 +216,17 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
premiere_timestamp = microformat.dig?("liveBroadcastDetails", "startTimestamp")
.try { |t| Time.parse_rfc3339(t.as_s) }
premiere_timestamp ||= player_response.dig?(
"playabilityStatus", "liveStreamability",
"liveStreamabilityRenderer", "offlineSlate",
"liveStreamOfflineSlateRenderer", "scheduledStartTime"
)
.try &.as_s.to_i64
.try { |t| Time.unix(t) }
live_now = microformat.dig?("liveBroadcastDetails", "isLiveNow")
.try &.as_bool || false
.try &.as_bool
live_now ||= video_details.dig?("isLive").try &.as_bool || false
post_live_dvr = video_details.dig?("isPostLiveDvr")
.try &.as_bool || false
@ -225,7 +237,7 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
.try &.as_a.map &.as_s || [] of String
allow_ratings = video_details["allowRatings"]?.try &.as_bool
family_friendly = microformat["isFamilySafe"].try &.as_bool
family_friendly = microformat["isFamilySafe"]?.try &.as_bool
is_listed = video_details["isCrawlable"]?.try &.as_bool
is_upcoming = video_details["isUpcoming"]?.try &.as_bool
@ -443,3 +455,35 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
return params
end
private def convert_url(fmt)
if cfr = fmt["signatureCipher"]?.try { |json| HTTP::Params.parse(json.as_s) }
sp = cfr["sp"]
url = URI.parse(cfr["url"])
params = url.query_params
LOGGER.debug("convert_url: Decoding '#{cfr}'")
unsig = DECRYPT_FUNCTION.try &.decrypt_signature(cfr["s"])
params[sp] = unsig if unsig
else
url = URI.parse(fmt["url"].as_s)
params = url.query_params
end
n = DECRYPT_FUNCTION.try &.decrypt_nsig(params["n"])
params["n"] = n if n
if token = CONFIG.po_token
params["pot"] = token
end
url.query_params = params
LOGGER.trace("convert_url: new url is '#{url}'")
return url.to_s
rescue ex
LOGGER.debug("convert_url: Error when parsing video URL")
LOGGER.trace(ex.inspect_with_backtrace)
return ""
end

View File

@ -2,6 +2,7 @@ struct VideoPreferences
include JSON::Serializable
property annotations : Bool
property preload : Bool
property autoplay : Bool
property comments : Array(String)
property continue : Bool
@ -28,6 +29,7 @@ end
def process_video_params(query, preferences)
annotations = query["iv_load_policy"]?.try &.to_i?
preload = query["preload"]?.try { |q| (q == "true" || q == "1").to_unsafe }
autoplay = query["autoplay"]?.try { |q| (q == "true" || q == "1").to_unsafe }
comments = query["comments"]?.try &.split(",").map(&.downcase)
continue = query["continue"]?.try { |q| (q == "true" || q == "1").to_unsafe }
@ -50,6 +52,7 @@ def process_video_params(query, preferences)
if preferences
# region ||= preferences.region
annotations ||= preferences.annotations.to_unsafe
preload ||= preferences.preload.to_unsafe
autoplay ||= preferences.autoplay.to_unsafe
comments ||= preferences.comments
continue ||= preferences.continue.to_unsafe
@ -70,6 +73,7 @@ def process_video_params(query, preferences)
end
annotations ||= CONFIG.default_user_preferences.annotations.to_unsafe
preload ||= CONFIG.default_user_preferences.preload.to_unsafe
autoplay ||= CONFIG.default_user_preferences.autoplay.to_unsafe
comments ||= CONFIG.default_user_preferences.comments
continue ||= CONFIG.default_user_preferences.continue.to_unsafe
@ -89,6 +93,7 @@ def process_video_params(query, preferences)
save_player_pos ||= CONFIG.default_user_preferences.save_player_pos.to_unsafe
annotations = annotations == 1
preload = preload == 1
autoplay = autoplay == 1
continue = continue == 1
continue_autoplay = continue_autoplay == 1
@ -128,6 +133,7 @@ def process_video_params(query, preferences)
params = VideoPreferences.new({
annotations: annotations,
preload: preload,
autoplay: autoplay,
comments: comments,
continue: continue,

View File

@ -1,5 +1,6 @@
<video style="outline:none;width:100%;background-color:#000" playsinline poster="<%= thumbnail %>"
id="player" class="on-video_player video-js player-style-<%= params.player_style %>"
preload="<% if params.preload %>auto<% else %>none<% end %>"
<% if params.autoplay %>autoplay<% end %>
<% if params.video_loop %>loop<% end %>
<% if params.controls %>controls<% end %>>

View File

@ -12,6 +12,11 @@
<input name="video_loop" id="video_loop" type="checkbox" <% if preferences.video_loop %>checked<% end %>>
</div>
<div class="pure-control-group">
<label for="preload"><%= translate(locale, "preferences_preload_label") %></label>
<input name="preload" id="preload" type="checkbox" <% if preferences.preload %>checked<% end %>>
</div>
<div class="pure-control-group">
<label for="autoplay"><%= translate(locale, "preferences_autoplay_label") %></label>
<input name="autoplay" id="autoplay" type="checkbox" <% if preferences.autoplay %>checked<% end %>>

View File

@ -1,3 +1,51 @@
# Mapping of subdomain => YoutubeConnectionPool
# This is needed as we may need to access arbitrary subdomains of ytimg
private YTIMG_POOLS = {} of String => YoutubeConnectionPool
struct YoutubeConnectionPool
property! url : URI
property! capacity : Int32
property! timeout : Float64
property pool : DB::Pool(HTTP::Client)
def initialize(url : URI, @capacity = 5, @timeout = 5.0)
@url = url
@pool = build_pool()
end
def client(&)
conn = pool.checkout
# Proxy needs to be reinstated every time we get a client from the pool
conn.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
begin
response = yield conn
rescue ex
conn.close
conn = make_client(url, force_resolve: true)
response = yield conn
ensure
pool.release(conn)
end
response
end
private def build_pool
options = DB::Pool::Options.new(
initial_pool_size: 0,
max_pool_size: capacity,
max_idle_pool_size: capacity,
checkout_timeout: timeout
)
DB::Pool(HTTP::Client).new(options) do
next make_client(url, force_resolve: true)
end
end
end
def add_yt_headers(request)
request.headers.delete("User-Agent") if request.headers["User-Agent"] == "Crystal"
request.headers["User-Agent"] ||= "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"
@ -13,56 +61,17 @@ def add_yt_headers(request)
end
end
struct YoutubeConnectionPool
property! url : URI
property! capacity : Int32
property! timeout : Float64
property pool : DB::Pool(HTTP::Client)
def initialize(url : URI, @capacity = 5, @timeout = 5.0)
@url = url
@pool = build_pool()
end
def client(&)
conn = pool.checkout
begin
response = yield conn
rescue ex
conn.close
conn = HTTP::Client.new(url)
conn.family = CONFIG.force_resolve
conn.family = Socket::Family::INET if conn.family == Socket::Family::UNSPEC
conn.before_request { |r| add_yt_headers(r) } if url.host == "www.youtube.com"
response = yield conn
ensure
pool.release(conn)
end
response
end
private def build_pool
DB::Pool(HTTP::Client).new(initial_pool_size: 0, max_pool_size: capacity, max_idle_pool_size: capacity, checkout_timeout: timeout) do
conn = HTTP::Client.new(url)
conn.family = CONFIG.force_resolve
conn.family = Socket::Family::INET if conn.family == Socket::Family::UNSPEC
conn.before_request { |r| add_yt_headers(r) } if url.host == "www.youtube.com"
conn
end
end
end
def make_client(url : URI, region = nil, force_resolve : Bool = false)
def make_client(url : URI, region = nil, force_resolve : Bool = false, force_youtube_headers : Bool = false)
client = HTTP::Client.new(url)
client.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
# Force the usage of a specific configured IP Family
if force_resolve
client.family = CONFIG.force_resolve
client.family = Socket::Family::INET if client.family == Socket::Family::UNSPEC
end
client.before_request { |r| add_yt_headers(r) } if url.host == "www.youtube.com"
client.before_request { |r| add_yt_headers(r) } if url.host.try &.ends_with?("youtube.com") || force_youtube_headers
client.read_timeout = 10.seconds
client.connect_timeout = 10.seconds
@ -70,10 +79,38 @@ def make_client(url : URI, region = nil, force_resolve : Bool = false)
end
def make_client(url : URI, region = nil, force_resolve : Bool = false, &)
client = make_client(url, region, force_resolve)
client = make_client(url, region, force_resolve: force_resolve)
begin
yield client
ensure
client.close
end
end
def make_configured_http_proxy_client
# This method is only called when configuration for an HTTP proxy are set
config_proxy = CONFIG.http_proxy.not_nil!
return HTTP::Proxy::Client.new(
config_proxy.host,
config_proxy.port,
username: config_proxy.user,
password: config_proxy.password,
)
end
# Fetches a HTTP pool for the specified subdomain of ytimg.com
#
# Creates a new one when the specified pool for the subdomain does not exist
def get_ytimg_pool(subdomain)
if pool = YTIMG_POOLS[subdomain]?
return pool
else
LOGGER.info("ytimg_pool: Creating a new HTTP pool for \"https://#{subdomain}.ytimg.com\"")
pool = YoutubeConnectionPool.new(URI.parse("https://#{subdomain}.ytimg.com"), capacity: CONFIG.pool_size)
YTIMG_POOLS[subdomain] = pool
return pool
end
end

View File

@ -21,6 +21,7 @@ private ITEM_PARSERS = {
Parsers::ItemSectionRendererParser,
Parsers::ContinuationItemRendererParser,
Parsers::HashtagRendererParser,
Parsers::LockupViewModelParser,
}
private alias InitialData = Hash(String, JSON::Any)
@ -108,21 +109,30 @@ private module Parsers
length_seconds = 0
end
live_now = false
premium = false
premiere_timestamp = item_contents.dig?("upcomingEventData", "startTime").try { |t| Time.unix(t.as_s.to_i64) }
badges = VideoBadges::None
item_contents["badges"]?.try &.as_a.each do |badge|
b = badge["metadataBadgeRenderer"]
case b["label"].as_s
when "LIVE NOW"
live_now = true
when "New", "4K", "CC"
# TODO
when "LIVE"
badges |= VideoBadges::LiveNow
when "New"
badges |= VideoBadges::New
when "4K"
badges |= VideoBadges::FourK
when "8K"
badges |= VideoBadges::EightK
when "VR180"
badges |= VideoBadges::VR180
when "360°"
badges |= VideoBadges::VR360
when "3D"
badges |= VideoBadges::ThreeD
when "CC"
badges |= VideoBadges::ClosedCaptions
when "Premium"
# TODO: Potentially available as item_contents["topStandaloneBadge"]["metadataBadgeRenderer"]
premium = true
badges |= VideoBadges::Premium
else nil # Ignore
end
end
@ -136,10 +146,9 @@ private module Parsers
views: view_count,
description_html: description_html,
length_seconds: length_seconds,
live_now: live_now,
premium: premium,
premiere_timestamp: premiere_timestamp,
author_verified: author_verified,
badges: badges,
})
end
@ -459,9 +468,9 @@ private module Parsers
# Parses an InnerTube richItemRenderer into a SearchVideo.
# Returns nil when the given object isn't a RichItemRenderer
#
# A richItemRenderer seems to be a simple wrapper for a videoRenderer, used
# by the result page for hashtags and for the podcast tab on channels.
# It is located inside a continuationItems container for hashtags.
# A richItemRenderer seems to be a simple wrapper for a various other types,
# used on the hashtags result page and the channel podcast tab. It is located
# itself inside a richGridRenderer container.
#
module RichItemRendererParser
def self.process(item : JSON::Any, author_fallback : AuthorFallback)
@ -474,6 +483,8 @@ private module Parsers
child = VideoRendererParser.process(item_contents, author_fallback)
child ||= ReelItemRendererParser.process(item_contents, author_fallback)
child ||= PlaylistRendererParser.process(item_contents, author_fallback)
child ||= LockupViewModelParser.process(item_contents, author_fallback)
child ||= ShortsLockupViewModelParser.process(item_contents, author_fallback)
return child
end
@ -488,6 +499,9 @@ private module Parsers
# reelItemRenderer items are used in the new (2022) channel layout,
# in the "shorts" tab.
#
# NOTE: As of 10/2024, it might have been fully replaced by shortsLockupViewModel
# TODO: Confirm that hypothesis
#
module ReelItemRendererParser
def self.process(item : JSON::Any, author_fallback : AuthorFallback)
if item_contents = item["reelItemRenderer"]?
@ -563,10 +577,138 @@ private module Parsers
views: view_count,
description_html: "",
length_seconds: duration,
live_now: false,
premium: false,
premiere_timestamp: Time.unix(0),
author_verified: false,
badges: VideoBadges::None,
})
end
def self.parser_name
return {{@type.name}}
end
end
# Parses an InnerTube lockupViewModel into a SearchPlaylist.
# Returns nil when the given object is not a lockupViewModel.
#
# This structure is present since November 2024 on the "podcasts" and
# "playlists" tabs of the channel page. It is usually encapsulated in either
# a richItemRenderer or a richGridRenderer.
#
module LockupViewModelParser
def self.process(item : JSON::Any, author_fallback : AuthorFallback)
if item_contents = item["lockupViewModel"]?
return self.parse(item_contents, author_fallback)
end
end
private def self.parse(item_contents, author_fallback)
playlist_id = item_contents["contentId"].as_s
thumbnail_view_model = item_contents.dig(
"contentImage", "collectionThumbnailViewModel",
"primaryThumbnail", "thumbnailViewModel"
)
thumbnail = thumbnail_view_model.dig("image", "sources", 0, "url").as_s
# This complicated sequences tries to extract the following data structure:
# "overlays": [{
# "thumbnailOverlayBadgeViewModel": {
# "thumbnailBadges": [{
# "thumbnailBadgeViewModel": {
# "text": "430 episodes",
# "badgeStyle": "THUMBNAIL_OVERLAY_BADGE_STYLE_DEFAULT"
# }
# }]
# }
# }]
#
# NOTE: this simplistic `.to_i` conversion might not work on larger
# playlists and hasn't been tested.
video_count = thumbnail_view_model.dig("overlays").as_a
.compact_map(&.dig?("thumbnailOverlayBadgeViewModel", "thumbnailBadges").try &.as_a)
.flatten
.find(nil, &.dig?("thumbnailBadgeViewModel", "text").try { |node|
{"episodes", "videos"}.any? { |str| node.as_s.ends_with?(str) }
})
.try &.dig("thumbnailBadgeViewModel", "text").as_s.to_i(strict: false)
metadata = item_contents.dig("metadata", "lockupMetadataViewModel")
title = metadata.dig("title", "content").as_s
# TODO: Retrieve "updated" info from metadata parts
# rows = metadata.dig("metadata", "contentMetadataViewModel", "metadataRows").as_a
# parts_text = rows.map(&.dig?("metadataParts", "text", "content").try &.as_s)
# One of these parts should contain a string like: "Updated 2 days ago"
# TODO: Maybe add a button to access the first video of the playlist?
# item_contents.dig("rendererContext", "commandContext", "onTap", "innertubeCommand", "watchEndpoint")
# Available fields: "videoId", "playlistId", "params"
return SearchPlaylist.new({
title: title,
id: playlist_id,
author: author_fallback.name,
ucid: author_fallback.id,
video_count: video_count || -1,
videos: [] of SearchPlaylistVideo,
thumbnail: thumbnail,
author_verified: false,
})
end
def self.parser_name
return {{@type.name}}
end
end
# Parses an InnerTube shortsLockupViewModel into a SearchVideo.
# Returns nil when the given object is not a shortsLockupViewModel.
#
# This structure is present since around October 2024 on the "shorts" tab of
# the channel page and likely replaces the reelItemRenderer structure. It is
# usually (always?) encapsulated in a richItemRenderer.
#
module ShortsLockupViewModelParser
def self.process(item : JSON::Any, author_fallback : AuthorFallback)
if item_contents = item["shortsLockupViewModel"]?
return self.parse(item_contents, author_fallback)
end
end
private def self.parse(item_contents, author_fallback)
# TODO: Maybe add support for "oardefault.jpg" thumbnails?
# thumbnail = item_contents.dig("thumbnail", "sources", 0, "url").as_s
# Gives: https://i.ytimg.com/vi/{video_id}/oardefault.jpg?...
video_id = item_contents.dig(
"onTap", "innertubeCommand", "reelWatchEndpoint", "videoId"
).as_s
title = item_contents.dig("overlayMetadata", "primaryText", "content").as_s
view_count = short_text_to_number(
item_contents.dig("overlayMetadata", "secondaryText", "content").as_s
)
# Approximate to one minute, as "shorts" generally don't exceed that.
# NOTE: The actual duration is not provided by Youtube anymore.
# TODO: Maybe use -1 as an error value and handle that on the frontend?
duration = 60_i32
SearchVideo.new({
title: title,
id: video_id,
author: author_fallback.name,
ucid: author_fallback.id,
published: Time.unix(0),
views: view_count,
description_html: "",
length_seconds: duration,
premiere_timestamp: Time.unix(0),
author_verified: false,
badges: VideoBadges::None,
})
end

View File

@ -111,7 +111,7 @@ module UrlSanitizer
new_uri.path = "/watch"
new_params = copy_params(unsafe_uri.query_params, :watch)
new_params["id"] = breadcrumbs[0]
new_params["v"] = breadcrumbs[0]
new_uri.query_params = new_params
end

View File

@ -29,6 +29,7 @@ module YoutubeAPI
WebEmbeddedPlayer
WebMobile
WebScreenEmbed
WebCreator
Android
AndroidEmbeddedPlayer
@ -80,6 +81,14 @@ module YoutubeAPI
os_version: WINDOWS_VERSION,
platform: "DESKTOP",
},
ClientType::WebCreator => {
name: "WEB_CREATOR",
name_proto: "62",
version: "1.20240918.03.00",
os_name: "Windows",
os_version: WINDOWS_VERSION,
platform: "DESKTOP",
},
# Android
@ -628,6 +637,11 @@ module YoutubeAPI
# Send the POST request
body = YT_POOL.client() do |client|
client.post(url, headers: headers, body: data.to_json) do |response|
if response.status_code != 200
raise InfoException.new("Error: non 200 status code. Youtube API returned \
status code #{response.status_code}. See <a href=\"https://docs.invidious.io/youtube-errors-explained/\"> \
https://docs.invidious.io/youtube-errors-explained/</a> for troubleshooting.")
end
self._decompress(response.body_io, response.headers["Content-Encoding"]?)
end
end