v0.5.6 - Performance Part 2 (Is that a new scan loop?)

For the past 3 months, I have been working tirelessly to rebuild our main scan loop, which is not only the most complicated part of Kavita but also the most critical. It translates files on your system into the series, volumes, and chapters you know in the UI. This goal of a new loop was so difficult, I quit 3 times, but finally made it through. The new loop is finally ready to share with you all, thanks to many of our nightly users helping test and provide feedback.

What is this new scan loop I speak of? The new scan loop is a new way (and a new set of requirements) to scan your disks and process files extremely efficiently. This changes Kavita to using a folder-based parsing mechanism rather than the previous mechanism where we would parse all files, group then perform DB operations in groups of 50. The reasoning for this drastic shift is that the old method failed to scale on users with massive libraries (we have one user with half a million files per stats) and would take an exceedingly long time for networked (rclone) users.

To dive in, we need to talk about some requirement changes. In order to utilize folder-based scanning, we have to remove the flexibility of users can put files anywhere they want and Kavita will group them for you. When I started, I chose this mechanism based on feedback from Plex and Jellyfin, but in reality most of our users have proper folder-based structures, so this new requirement change shouldn't affect many users.

Kavita expects all series to be in their own folder, so a library cannot have loose leaf files in it. You can still do /library root/manga/Publisher/Series A/, /library root/manga/Publisher/Series B/, just not /library root/my book.pdf. Kavita will now scan folder by folder an for each folder, group the files in and process them as a single series in a task (async). This task will do all DB operations and invoke Cover Gen and File Analysis (word counting) for just that series. This change is important because it will now process series-by-series and update your UI in that manner. Hence you should be able to more quickly see your series and use them completely than before on first scan. This however comes with a drawback due to increased I/O and trips to the DB, the library scan is slower (on first scan). For me, previously it took 30 mins to scan all files into the DB (without covers or word counts), on the new loop it takes 80 mins (but has covers and word counts). The main difference is on the old loop I had to wait at least 10 mins till the first chunk inserts, with the new loop, you don't have that issue. This also means the Chunk errors users sometimes receive should no longer be an issue and if there is an error, that series will tell you directly. You can find more information about the new Scan loop on the wiki.

With that said, this also allows some deep optimizations. Kavita will no longer do any I/O (except an attribute lookup on folders) if a folder hasn't changed since the last scan time. This single optimization means that my library scan can run in 5 seconds for my 660 series library, as it skips work on anything that hasn't changed. From the nightly testers, many users have seen drastic time reduction, esp our rclone users. This also allows for 2 enhancements, the ability to ignore certain files/folders (aka .kavitaignore) and the ability to hook into Kavita and say a folder has changed and let Kavita kick off a scan.

Next up is .kavitaignore, inspired from .plexignore and a feature request as well, users can now tell Kavita to ignore files and folders at any level of a scan. You can have at library root to have a global set of rules, like ignore cover.png, or you can have nested into your folders. You can have one for each folder and Kavita will intelligently combine the rules as it moves down the folder tree. This is another degree of freedom which allows you to not only filter, but speed up scans as well. This feature is experimental and still being tested and tweaked, but works in many cases. Give it a go and report back on issues faced or clever tricks that we can enhance our wiki with.

Lastly, we have folder watching. This new, experimental feature is really something cool. Why wait for nightly scans, when Kavita can pickup on changes and instantly run an appropriate scan? This new feature does just that. When a change is detected, a task is queued up and after 5 minutes executed. Any changes that related to that first task, will get disregarded. Please note, this is experimental and likely to not work if you download directly to your library. This feature will be fletched out further in later releases.

TLDR: There are new requirements for what Kavita expects from file layout. Read about it on the wiki. If you scan without conforming your layout, you may run into issues.

If you have been enjoying Kavita, please consider donating or backing me. Kavita is primarily built by just me in my spare time and each release is hundreds of hours of work.

Added

• Added: You can now have a .kavitaignore file at any location in your library and the scanner will read that and filter out any subsequent folder/files. This works very similar to .plexignore, this utilizes a Globbing syntax and supports multiple patterns via new lines. You can find more about the syntax here. This feature is experimental
• Added: Added a new event type of Info which will show when Scan loops are aborted due to nothing changing on disk. This aims to inform the user that their expected action took no result.
• Added: Added a clear all button on the events widget
• Added: When a new series is added to Kavita, library detail pages will insert instantly (previously we reloaded the page every 3 seconds post an update coming in)
• Added: Added --event-widget-info-bg-color for the new info event widget color
• Added: When we navigate from a page that has a jump bar and the user has clicked the jump bar, resume their scroll position when returning to that page. This is not tracking scroll position naturally, that will be looked at later.
• Added: Added / as a server route to swagger, which could help users behind a reverse proxy from using their local server.
• Added: Kavita now comes out of the box with Folder Watching for libraries. This means that when any sort of event occurs, like new files, file/folder renames, deletions, etc, Kavita will queue a task to scan those changes into the system. By default, tasks will only be processed every 5 mins. This is an experimental feature and disabled by default.
• Added: Kavita now comes with a new API for library/scan-folder and takes the API key (with admin permissions) and a folder path.
• Added: Added API endpoint for a health check to streamline docker healthy status.

Changed

• Changed: Kavita now requires series to be within a folder and not spread between multiple folders. For example, a library root of /manga/ must have series self contained in /manga/series A/files, /manga/series B/files. Kavita will now ignore files that are in /manga/ alone.
• Changed: Rewrote the scan loop to perform folder-based scanning, Kavita will now process one folder at a time in a multi-threaded manner. This is a major change to how previously scanned, where all files had to be parsed before any DB work could be done. Users on networked systems should see a major improvement.
• Changed: Scanning tasks now run on their own dedicated queue and will not allow any concurrent runs. Time for a scan task to get executed is lengthened a bit as default queue takes priority.
• Changed: Added optimizations where Kavita will ignore scanning folders (all I/O) if nothing has changed on the disk since last scan of that folder.
• Changed: Fail silently if an email invite doesn't get sent, as the link is still present in the DB and UI screen.
• Changed: Ignore progress events altogether on Series cards so progress bars don't get skewed when a user is reading
• Changed: Reduced the spammy output for the logger, so more clear information is delivered, instead of internal DB queries.
• Changed: Deleting a series will now automatically remove it from the Want to Read page
• Changed: Tweaked a migration to output log only if it's going to be ran
• Changed: Added dedicated token output to logs for auth flows (when inviting a user) in case token is getting url encoded (but from testing, it doesn't affect)
• Changed: Reverted back to ES6 vs ES2020 so older Safari browsers can use Kavita
• Changed: When parsing non-epubs in Book library, use Manga parsing instead of Comic, to better support Light Novels
• Changed: OPDS description now no longer mentions users will be able to download, as they currently still need the download role.
• Changed: Updated all links that point to external sites to use noopener noreferrer
• Changed: Updated the Email Service wording to point to the Github for the external email service
• Changed: Clear cache will now clear temp directory as well, to clear out bookmark and downloads.
• Changed: When a library is in progress and a user tries to delete the library, tell them to wait before we let them
• Changed: Rearranged some code in invite user flow to help with the small fraction of users that can't invite users.
• Changed: Added a null check so we don't log an exception when connecting a websocket when user is not authenticated
• Changed: When a theme that is set for a user, gets removed from Kavita, inform the user to refresh.
• Changed: Manga reader Darkness control is now called Brightness

Fixed

• Fixed: Fixed a bug where series with LocalizedSeries comicinfo tag could ...

v0.5.5 - Performance Part 1

This is another release where I forgot just how much I did. The intent of this release was performance and held tons of work that I had been wanting to tackle, but needed the dedicated time for. Originally I had planned to release this with the new scan loop, however due to security vulnerabilities and some bugs in the hotfix, I had to split the release in half. As mentioned, the focus is on performance and we are bringing a ton of nice performance improvements along with a new contributor, @TheIceCreamTroll, who not only opened a lot of bugs on Github, but also helped fix a few in the process. I look forward to more development work from them.

To start off this highlight, comes Downloads. Downloads have always been pretty reliable but when you're downloading tons of content for an upcoming trip, it can first get annoying when you leave the page, you might loose the download. Also that downloads kept prompting you for confirmation when the files are large (to hopefully not eat up your data cap). So this release, I reworked downloads significantly. Downloads are now handled at a global level, range processing is enabled (this means you can stop/start if your browser supports it), downloads are cached and thus streamed to your browser (less memory overhead), the event widget handling has been tweaked a lot to make it much more streamlined, and my favorite, if you start a download, go to another page then return, the download progress bar on the card will still show.

Another huge update for Kavita is moving to OnPush change detection, something that is more technical and more work for me, but has nice performance implications for the user. OnPush basically means that the code tells Angular when it's time to re-render, which means less work Angular does and hence more frames per second. Pairing this with the update to Angular 14, the UI should feel a lot snappier to you.

This next one is my favorite. I was shocked when I was on a non-admin account and realized, I didn't have a way to track what I wanted to read...normally, I would create a collection but then I realized, Kavita's collections are for admin's only (modeled after Plex originally). So I sought out to build a Want to Read list that is like a collection that tracks what you want to read. Right now, we have ability to filter against it and that's it. I plan to build it out in future updates; if you have any ideas, jump on the feature request site and submit them.

In addition to performance, I also focused on polishing up the double page reader from last release and fixed an annoying infinite redirection bug, usually seen on Firefox. If you have issues with the double page render, def stop by or create a Github issue to let me know.

Lastly, a few users have been having issues with the invite user flow. I've been hard at work trying to support these users, but have been unable to reproduce their issues. I made some changes in the Invite User flow to hopefully improve their lives, but behind the scenes I'm still engaged in support and planning enhancements around this Authentication system. Thank you for being patient. Early reports from users seem to have less issues post these changes, so give it another go if you fell into this bucket.

Lastly is a call to the community if anyone is interested in helping in Support in our discord or being a dedicated release tester. Kavita has gotten to the size where it is taxing to be building and testing all by myself. Currently release testing takes 3 days split between 3 people. I really need help from the community to help keep Kavita stable and let me focus on development rather than Support and Testing. If anyone is interested, please reach out to me in the discord.

With all that said, the next release will be even slower than this one. I have a lot of vacation planned for next 2 months and the left over work is extremely difficult, as it's the most complicated and critical part of Kavita. Thank you as always for being patient with releases and for those that have donated, I am extremely grateful. I've been buying my Kavita supplies (lots of coffee to get me through the scan loop).

Note: The security vulnerabilities may not be viewable till a few days after this release, to allow all platforms to upgrade their builds. Anything listed has been validated and fixed as of this release.

Added

• Added: Added new event hooks for when mark as read/unread occur on entities, which will now trigger a re-calculation on Read Time Left.
• Added: Cover Image uploading will now allow gif and webp files
• Added: When the user is requesting background jobs, like scan library, Kavita will now disregard jobs if there is already a job in the queue for that configuration.
• Added: Downloads now support range processing (the browser can theoretically resume the download)
• Added: Downloads will now persist through page loads and the indicator will show on any card/list items where applicable.
• Added: Downloads are now global and a notification in event widget helps you see the download progress (browser progress).
• Added: New user preference to prompt for downloads larger than 100MB. When disabled, Kavita will download without prompting.
• Added: Downloads will now temp be left on disk and thus if requested again, can send the same file without re-generating it.
• Added: New placeholder for when covers haven't loaded in yet.
• Added: PDF Reader now has a loading indicator with a progress bar to indicate how much of the PDF has been downloaded by the browser.
• Added: PDF Reader will now close itself when ESC is pressed
• Added: Added the ability to manually setup users without having to worry about the invite url during invite flow or doing it right at invite time. Now a Setup button will appear next to all Pending invites and at any time can be pressed to manually complete the account setup. Note: This does not work with pending invites created before this update. Please re-create them to get the new functionality.
• Added: Added back to top support on all pages but those that utilize virtual scrolling without a parent scroll
• Added: Pressing H will open the shortcut modal
• Added: All users will now have a dedicated List called 'Want to Read' which will allow them to track series they are interested in reading. They can apply filtering against this page.
• Added: Added a quick way to set darkness level on manga reader for when night light just isn't dark enough (this is not a setting, it is being trailed and subject to removal)
• Added: Added Japanese Series name parsing support in Parser
• Added: Added Jump bar to reading list page

Changed

• Changed: Changed padding to allow for two cards side-by-side on viewport width of 375px.
• Changed: Changed every component to use OnPush Change Detection, resulting in less wasted render work and a snappier experience overall
• Changed: Moved some calculations for title and subtitle in manga reader to the backend
• Changed: Bulk Operations bar now has an explicit mark as read/unread buttons.
• Changed: Bulk Operations now has Add to Collection option and Delete where appropriate (and for appropriate users)
• Changed: Edit Series Relation will focus on the next typeahead after pressing Add Relation
• Changed: Cover Image chooser will now show the chapter's cover images as options by default
• Changed: Added more aggressive static file caching
• Changed: Refactored reading list item component to drastically reduce the amount or render calls. Huge performance for users with large reading lists
• Changed: Typeahead control now has an animation for opening and closing
• Changed: Scan Series, when invoked via UI/API, will force a re-calculation of metadata and ignore any caching, to ensure reliable results.
• Changed: Added Range processing support for images on the reader for slower networks or large files
• Changed: (Manga Reader) When using single mode, try to use a pre-fetched Image, rather than relying on browser to cache the image
• Changed: Reduced some latency when rendering first page of next chapter via continuous reading mode
• Changed: Downloads will now leave the file on disk and re-use them. These files are cleaned up nightly or after any scan (scan series/library)
• Changed: When a user is downloading a file on Kavita, admin's received a notification about it. This notification is now reworded to imply that the server is processing the download. This notification will be removed in future version.
• Changed: Changed how caching works within Kavita. Cover Images are now properly cached for 1 min, Images for the reader are cached for 10 mins, and the pdf file (for new reader) is also 10 mins. This has a known issue where updating a cover image, the browser wont reflect the new image on Chrome based browsers. This is due to Chrome not respecting the cache-control spec. A hard refresh will fix it.
• Changed: (Performance) Applied SplitQuery to multiple queries throughout the codebase, speeding up many API calls.
• Changed: Exception middleware will now send the original error message to the UI rather than a generic 'Internal Server Error'.
• Changed: Error toasts now have Error in the title along with the status code
• Changed: Changed volume titles in list view to include volume number.
• Changed: Implemented a workaround for nginx users with BlockCommonExploits enabled, which would interfere with book image escaping done by Kavita when images had ../ in their path.
• Changed: Hide jumpbar on pages where there is no scroll
• Changed: Non-admins can no longer view file info on card detail drawer
• Changed: Typeaheads will no longer close after selecting an item
• Changed: If there is no collection or reading list cover image, hide the image placeholder
• Changed: Optimized bookmarking a page by reducing a DB trip
• Changed: Search bar now will only show clear button once a single character is t...

v0.5.4.2 - Security Hotfix v2

This is a hotfix to the hotfix, fixing image loading within Kavita.

Fixed

• Fixed: Fixed a bug with new authentication where images and pdfs would fail to load

Known Issues

• Tachiyomi no longer connecting
• Collection image covers are not showing on Collections Page (but collection detail it does show)

v0.5.4.1 - Security Hotfix

This is a security hotfix, please update immediately! Several high severity issues were disclosed to the Kavita team this morning that require an immediate hotfix.

Thanks @vultza for disclosing these issues through Huntr.

Added

• Added: After 5 login attempts, Kavita will impose a 10 min lockout before authentication can be retried

Fixed

• Fixed: Fixed an exploit that allowed a user to change the admin user's account password, thus taking ownership of the server
• Fixed: Fixed an exploit that would allow an unauthorized user to delete reading list items within a user's reading list
• Fixed: Fixed an exploit that allowed a user to abuse an API and leak sensitive files from the system
• Fixed: Fixed an exploit that allowed an unauthenticated user to download pdfs from the server
• Fixed: Fixed an exploit (SSRF) that would allow a user to upload a cover image that isn't an image

v0.5.4 - Double Page Layout, Estimated Reading Time, a new PDF Reader and Infinite Scroll?!

This is a hell of an update, so big I could barely figure out what features to put in the header. There were so many great additions that were not planned for this release, like the new PDF reader. Welcome to v0.5.4, this release is the part 2 of the UX and Reader refresh. This is another massive release, even I'm shocked when I was creating the release notes. This releases adds some big hitters and some requests from the community. Let's get started.

Coming up first is a niche feature for myself, but a few of our users will definitely like this, WebP storage support for bookmarks. If you use the bookmarking feature, your bookmark collection can get pretty big pretty fast. This new feature allows Kavita to covert images to WebP and save 40% space in the process. This is some foundational work as I want to expand this for all covers as well (WebP is missing Safari support).

Estimated Reading Time
This is a really cool feature as it helps you find what to read next based on how much time it might take to read it. Want something quick to read, your dashboard now shows series that it takes under 10 hours to read. If the series is image based, we use some estimates taken from our discord community to estimate a range of how long it takes to read. If it's a book, same deal, but we use word count rather than pages. Of course, estimated read time is now available for sorting as well. See some examples below:

List View
This is another one that is pretty cool and what looks like a simple feature, is in fact a lot of work under the hood. This new layout mode for the site really shines for book readers. Not only can you see the titles (if they are set), but also summary and other information, making it really easy to pick up a book to read. Plus, combined (in this screenshot) is the new Blur Unread summaries setting, which will blur summaries if you haven't read them, so you wont be spoiled when glancing over books in a series (like I was, creating this feature).

PDF Reader
I've always been annoyed with the PDF reader Kavita came with out of the box, but I personally don't have many PDF files, so it's not a big deal for me. I've been monitoring and searching for libraries that would provide a better experience and low and behold, I found one. It's not perfect, but it was easy to integrate and gives a much more natural experience when reading PDFs. I would love to hear feedback on this new reader from you, so drop by the discord or open a feature request with some ideas for it. This is our v1 implementation so some more reading focused features are not present. Feed me ideas so we can build out a more ideal solution.

Infinite Scrolling This is a big one. There is no more pages in the app (except a few places that haven't been attended to), everything is infinite scrolling with a jump bar to quickly jump around and virtualized DOM to keep it light on the browser. Due to the infinite scroll, the scroll bars are in different places than you might be used to. It might be jarring at first, but you'll quickly get used to it and love the speed at which the screen renders.

Double Page Layout
This is another big one. I had implemented most of it in Part 1, but had to pull from the release as it had some edge cases. @magujun took it upon themselves to drive it home and it's simply sweet. If you're a big comic fan or even want to read emulating a comic, this is the feature for you (on appropriately sized devices). We offer 2 new double page layout methods: Double and Double (manga). Double will render out the pages as Page 1 Page 2 | Page 3 Page 4. Double (manga) renders out the pages as Page 2 Page 1 | Page 4 Page 3 to help mimic the reading order of manga.

Detail Drawers
This is the last big feature to talk about and that is moving to drawers rather than modals for some screens. This new UX pattern is me playing with the idea and testing it out. So far, it seems to be much nicer experience on mobile.

With those big features out of the way, there are still plenty of great enhancements in here, like the ability to mark chapters as specials via ComicInfo.xml, parser support for Chinese, Korean, and Japanese characters, a redesigned cover upload, ability to enable swagger on your server (for those who develop scripts against our API), and even a redesigned Admin panel, to drive some future updates.

Next release will be another slow release, especially since I'm doing some deep refactors of the main scan loop. Next release is focused solely on performance and scan loop enhancements and will take quite some time. Thank you as always for being patient and supporting me in this project. Especially to those that provide support in discord and our donators.

Added

• Added: New Server setting to use WebP for bookmarks rather than the raw image. WebP provides a 40% saving to space. This setting will take effect but not convert existing bookmarks (use task for that)
• Added: New Adhoc Task to convert existing bookmarks to WebP. The setting to use WebP does not have to be enabled. This can be a long running task, esp on ARM, so be aware of that.
• Added: When user account updates occur, like changing permissions or library updates, events will be sent to that user if they are logged in/active on the site and patch their new state in. This ensures they don't need to log out and log in to have the new state
• Added: Exposed internal reoccurring tasks to the UI under Tasks tab. This is preliminary work for a later update.
• Added: Kavita can now read the Format tag from ComicInfo and use that to force special status. The following tags will force special: Special, Reference, Director's Cut, Box Set, Box-Set, Annual, Anthology, Epilogue, One Shot, One-Shot, Prologue, TPB, Trade Paper Back, Omnibus, Compendium, Absolute, Graphic Novel, GN, FCBD
• Added: Implemented the ability to parse some volume and chapter keywords for Chinese
• Added: Added a word count analysis to count number of words in your epub files.
• Added: Added an time estimate to read a series based on average reading time for comics/manga and text. Time to read is powered by pages for comics/manga and words for epub
• Added: Implemented file parsing support for Korean
• Added: Series detail page will now show an estimated time left, if you have reading progress, based on an average speed between our fast and slow readers.
• Added: Library Recommendation page now has a new feed, Quick Catchups. Series that take less than 10 hours and are ongoing.
• Added: On Volume details, added a distinct Read button, rather than having hidden knowledge of clicking the poster for the chapter would let the user read it.
• Added: Series Detail has a new button in the top right which lets you use a new layout mode for laying out cards (without having to change user preferences).
• Added: List view layout. This new layout uses a list style to render out non-series on series detail page. There is more information on these cards by default and can show the title of the chapter/issue/book.
• Added: Swapped out the pagination for infinite scroll and jump bar to quickly jump around library
• Added: Added Download action to series detail page
• Added: Implemented the ability to use double page layout in the manga reader. This new rendering mode allows for both Comic (Page 0 Page 1 | Page 2 Page 3) book reader and Manga (Page 1 Page 0 | Page 3 Page 2) reading and is optimized for larger screens. Shoutout to @magujun for finishing off the implementation and getting it into this release.
• Added: Brand new PDF reader that is more like the native reader in browsers. PDF files will be streamed to the UI (this means way faster load times) and have ability to search text within, select text, zoom, change reading layout, etc. Progress reporting is hooked in. Light/Dark mode is supported but experimental in underlying library, use at your own risk.
• Added: When a series is removed form a library, library detail will now reload.
• Added: Filename parser now supports Japanese kanji for volume and chapter parsing
• Added: Admins can now enable swagger (exposed via swagger/) on their instances via a new server setting.
• Added: New server setting to configure how many days of backups are retained. Still defaults to 30 days.
• Added: Added the ability to sort by time to read from metadata filter
• Added: When there are errors extracting archives, throw a custom error message to the UI informing of the potential issues
• Added: Admins can now change a library type after creation. There is a warning explaining that it's very likely series will be re-created and hence reading progress or bookmarks could be deleted. Proceed with caution.
• Added: Users can now enable a setting to blur summaries (mainly for list view) of volumes/chapters/books they haven't read yet.
• Added: Added new stats to help see how features are being used and gather some performance testing numbers from v0.5.3 and to prepare for upcoming performance release (v0.5.5). In addition, some of the new functionality (virtual scrolling) have new stats to ensure ...

v0.5.3 - Book Reader Enhancements, Related Series, and More!

v0.5.3 is here and packed full of a collection of random features I wanted to build out for Kavita. I usually plan out the next few releases and stick to that schedule, adding in features that I want or users suggest randomly, but this time I wanted to sit down and take a release for myself to work on whatever I felt like, rather than what I had planned and it turned out to be really fun and some great additions came from it.

Let's start with what the discord community is abuzz about and that's the fated book reader refresh. Started last release but pushed due to complexity was a big refresh to the UX around the book reader and bringing one of our most up-voted feature request, calibre-like reading. Not only did we bring this reading mode to Kavita's reader, we also brought along new color themes for the reader for our OLED users, a font for Dyslexic users, and immersive mode, which hides the menuing system and lets you focus just on your book (goes great with column layout). It's been great iterating on the book reader since it was first built. I can't wait to see how it evolves and is polished further in later releases.

Next is my personal favorite and another coming from feature request, Series relations. Isn't it really annoying reading a series and not knowing what the next series is or what spin-offs exist? Well, now you can configure this into Kavita. From any series you can configure relationship links to other series, like sequels, prequels, spin off, side story, alternative version, etc. There are many links to choose from. You can see a little teaser of it below. Jump on our wiki to learn more about the feature.

Another killer addition is a big update to our search. You spoke and I heard you, you can now search for individual titles (book or manga/comics) and for individual files in addition to series, genres, people, tags, reading lists, libraries, collections, etc. This was tricky to implement and still retain the non-functional requirement of everything responding under 300ms. There should be no issues with you finding things in your server.

Wrapping it up is a few small but powerful ones. Likely you've seen it a few times in your logs, but the old constraint issue. I finally was able to track it down and squash it for good. It ended up being an index in the database that wasn't actually being used. Another one is the extra white space on the right side of the card grid system. Robbie was able to drive a beautiful solution and it works just as I envisioned. Lastly, On Deck was tweaked yet again to try to achieve a good balance of how I want it vs how I can actually code it and we have found a good compromise that works.

As usual, the release is packed with many things, so read through the changelog. To close off, I want to say thank you for all the support and suggestions to help shape Kavita. I have a lot of ideas that I want to see realized and really appreciate the inputs that have helped us build out Kavita.

Note: Kavita releases will slow down for a while as I have taken a new job. I am still committed to bringing my vision of Kavita to life, so don't worry about this project dying. I use Kavita daily.

Added

• Added: Users can now start reading from any chapter within a Reading List
• Added: Reading lists now have Cover images like Collection tags (https://feats.kavitareader.com/posts/89/adding-cover-images-to-reading-lists)
• Added: Added ability to filter series on a collection page
• Added: Library detail, all series, and collection page filtering will now persist in the url. This allows you to make a filter, click a series and hit back in browser to load the screen as it was, on the same page with the same filter as previously.
• Added: Card's progress bars will now automatically reflect a users' reading status in real time as a user is reading in real time without refreshing
• Added: Added a new Sort by field on filtering, Last Chapter Added, to help users sort in a way that mimics On Deck
• Added: Edit Series modal information tab will now show Last Chapter Added and Last Read Progress
• Added: Added a link to the wiki for stat collection to help users understand exactly what we collect on the setting field
• Added: GIF image types are now supported
• Added: Bookmarks are now moved out of user settings and are their own page on side nav.
• Added: Bookmarks can now be read by clicking on the series they are from, which will open the manga reader for that set of bookmarks.
• Added: Users can now link series together via Edit Series modal. Linked series will show on Series detail page under the Related tab.
• Added: Kavita will now read and group based on the LocalizedSeries tag within ComicInfo. This is not an official tag, but an issue is raised on Anansi Project. This tag maps to Series LocalizedName field.
• Added: Library Detail now has 2 tabs: Library and Recommendation. Library is your library and recommendation is detailed below.
• Added: Library Detail Recommendation tab now has actual recommendations. A first pass, we now show (for the current library only): On Deck, Quick Reads (series that are not read, not ongoing, and are under 4k pages), Highly Rated (series rated by any user on the server with more than 4 stars, sorted by average star count), Rediscover (series that you're already read), and lastly More In (more in a given genre).
• Added: Users can now search directly for files within Kavita. Clicking on the file will open the respective series.
• Added: Users can now search against Chapters with TitleName set. TitleName is extracted from the Title field in ComicInfo or the book name from Epubs (i.e. if you search 'Stone' and have a series of Harry Potter, 'Harry Potter and the Sorcerer's Stone' will show up as it's the 2nd book in the series)
• Added: Added total pages to series info modal
• Added: When a file doesn't exist on disk but still in Kavita and the user tries to read it, throw an error and show it on the UI rather than just spin forever.
• Added: New book reader drawer design. Drawer has been redesigned to break up settings and table of content to reduce the amount of scrolling and to provide a cleaner UX.
• Added: Column layout (1 or 2 column) is now available in the book reader. This is a no-scroll solution which works best on e-ink readers.
• Added: New coloring themes for the book reader. Users can now choose between Dark (existing), Light (existing), or Black (new, all black)
• Added: Blacklist .qpkg folders and don't scan Recently-Snapshot or recycle folders.
• Added: New OpenDyslexic font for the book reader
• Added: You can now hide the action bar (top/bottom bar) in the book reader by using Interactive Mode. Interactive mode will automatically turn on tap to paginate and gives you a distraction free way to read. In order to show the drawer and top bar, you can tap in the document on a non-interactive element.
• Added: Added --bulk-selection-highlight-text-color and --bulk-selection-text-color for bulk selection bar.
• Added: Added --search-result-text-lite-color for search results accent (where library is shown)

Changed

• Changed: Aligned the edit collection tag modal to align with new tab design
• Changed: Small change in webtoon reader to try to avoid using max width on PC's
• Changed: (Performance) Removed some nested loops and avoided some list generation in scan loop
• Changed: When opening a pre-filtered page, the filter section won't automatically open. Instead, the filter icon will be green showing that a filter is in effect.
• Changed: Made some style changes in the manga reader to better center images in the reader on different reading modes
• Changed: When a special has a Title set in the metadata, on series detail page show that on the card rather than filename
• Changed: Increased Recently Updated to give back 30 series worth of additions while still hitting our Non-Functional performance requirements
• Changed: From dashboard, clicking any activity stream (carousel)'s title, will go to all series page with a custom filter to emulate the underlying stream.
• Changed: Tweaked the styles on image cover chooser
• Changed: Typeahead filters are now disabled when they can't be changed. For example, library page, you can't remove the library from the typeahead.
• Changed: Event widget can now scroll vertically if there are messages that go past the viewport height.
• Changed: When opening the side nav on mobile with metadata filter already open, collapse the filter.
• Changed: When on mobile viewports, when clicking an item from side nav, collapse it afterwards
• Changed: Updated vers-one/EpubReader library which will now allow for reading epubs with bad table of contents (missing items), something that a lot of users complained about.
• Changed: When an epub doesn't have a dc:date set with publication event type, default to the dc:date tag for publication date.
• Changed: Updated ComicInfo MA15+ to MA 15+ per Anansi Project (wrong info was documented originally)
• Changed: Enhanced cover image selection such that back_cover and back-cover will no longer be selected
• Changed: Large Scale modularization to cleanup our lazy loaded modules and implement prefetching lazy loaded modules. The app should load even faster now.
• Changed: Home page (dashboard) is now called libraries/ instead of library/
• Changed: Removed cleaning some edition tags from Series name, like Full Color Edition or Full Contact Edition.
• Changed: Adjusted some styling in the search results screen
• Chan...

Hotfix 0.5.2.5 - PDF Rendering Regression on Pi 32-bit

It has come to my attention that PDF rendering broke on 32-bit Pis due to a downstream dependency and a .net regression. The downstream library has put a workaround and we have confirmed it works in this. Sorry for the inconvenience. This will be the last hotfix.

Added

• Added: Backup now includes your themes folder.

Fixed

• Fixed: Fixed an issue where PDFs would not render on 32-bit Pi's due to a dependency having a bug.
• Fixed: Fixed a copy issue where sometimes non-existent files would cause existent files to not copy during backup
• Fixed: Fixed a bug where bottom action bar on the book reader would have side-effects that top didn't, like settings button wouldn't open the drawer.

Hotfix 0.5.2.3 - SortName & Invite User Redirect fix

There were some unfortunate issues that slipped through release testing, so please update. The main focus for this hotfix is around SortName on series, which was getting cleared during a scan and not repopulated, thus breaking the sorting logic within Kavita.

In addition, some issues were found around Invite user, where unauthenticated browsers would not properly show the registration screen. These and a few more bugs reported have been fixed.

Please note, On Deck is still not functioning as expected from previous release and in this hotfix. It is planned to get a full treatment to align to my vision. Please make due with it as is.

Note 2: The hotfix is skipping patch .1 and .2 in order to better align with our nightly users.

Changed

• Changed: Slightly tweaked the logic of On Deck to consider chapters in a series that have been created within 30 day cutoff in addition to having been read. This does not fix the inherit issues with it, but serves as a first start.

Fixed

• Fixed: Fixed a bug where some events weren't being sent to admin clients only
• Fixed: Fixed a bug where on scan, if SeriesSort wasn't set in ComicInfo, it would get cleared, thus breaking sorting on library detail page
• Fixed: Fixed an issue where when falling back to folder parsing, sometimes when the folder name wouldn't parse well, like 'Foo 500' which parses as 'Foo'. Now the fallback will check if we have a solid series parsed from filename before we attempt to parse a folder.
• Fixed: Ensure SortName is always being set during a scan loop, even if locked, to ensure that it's never just an empty string (which breaks sorting)
• Fixed: Fixed some errors being thrown from metadata update for files that haven't had a summary set.
• Fixed: Fixed a bug where updating a series name to an existing name in the system wouldn't throw an error
• Fixed: Hide the side nav hamburger toggle when not authenticated
• Fixed: Fixed some logic issues around determining which releases are installed, available, etc from Announcements page.
• Fixed: On Firefox, upload cover image via url had different event properties than other browsers and would completely fail.
• Fixed: Confirm email (invitation link) would redirect to login when not authenticated due to side nav having an authenticated call in it.

v0.5.2 - UX Refresh, Custom Themes, Metadata Editing, and OPDS updates!

This was another huge release. I knew it was going to be big going in, I didn't expect that I'd had to split some of the features out into a separate release. This release is themed around UX enhancements and as you can see, we added custom theme support to Kavita (which was a lot of code changes behind the scenes). Admins can now either use one of our themes (eink is a new one) or build their own and encourage everyone to use it by setting it as the default theme for new accounts. Users can always change their theme based on their own needs.

We also added a side nav to Kavita. This was always my plan from day 1, but coding a side nav is no fun, so I had to wait a while till @therobbiedavis was here to do the heavy lifting. Thanks again Robbie! The side nav removes the need to have the libraries on the main page and based on my and the nightly users testing, is a huge experience upgrade. I'm excited to see what you think of it.

Another big feature is Metadata editing, something I've been looking to get to since we added metadata. Admins can now edit metadata in the UI and lock fields so that metadata updates from files do not override the custom edits done. This also allows to mark a series as 'On Hiatus', something that isn't possible with ComicInfo.xml. The edits stay in Kavita's DB. I will be assessing saving to the underlying file at a later time.

One more big feature that's not in the release title but should be is that the events widget that shows when a scan is in progress has been rewritten in a major effort to send more detailed information to the UI and ensures that only admins receive the messages when they should be protected by RBS. The widget will now show filenames when being scanned, series when they are updated or being generated for covers, and even errors that might occur, like you have an empty base folder from the library, which indicates a mounting problem. This is a really big enhancement, great to see, esp if you have a larger library or use networked storage.

Lastly, to wrap up. I finally got to re-tackle OPDS and help flatten some of the clicks users have to do and provide some better naming, which mimics the new series detail page rolled out last release (but also enhanced in this release). Kavita will now flatten based on some of the conditions of your series to reduce clicks. Search has been updated, download names as well. I hope the OPDS users enjoy it and let me know if you have ideas to improve it.

This release was a blast to work on and has many changes I can't touch on, please read through the changelog. I was lucky to be able to fix or improve a lot of different areas and features in Kavita thanks to the scope of changes the theming update required.

Some of my favorite smaller changes this release are:

1. The Cover Chooser now works with urls pretty consistently
2. Updating the Manga reader to use raw images giving us a much better experience on smaller screens with 'Fit to Screen' splitting mode
3. Huge performance gains on Series Detail page
4. Enhanced stat collection of what features users are using, like collections, reading lists, etc. I'm really excited to see what people are using so I can focus my effort in those areas.

Known Issue: Currently On Deck is not working as I want it, sometimes series will not get included on it, when they should be. I have plans to do a rewrite of this functionality for the next release. Please bear with me until then.

Added

• Added: Introducing custom theme support! Admins can now add their own custom theme files and have Kavita use them as the default for new users. Users can change their own site theme to any on the server, including Kavita's default themes. Admins can also default all users to any theme, including a custom one. See more about theme's here.
• Added: Added a hover effect to card to give more feedback to the user.
• Added: Kavita now comes with an e-ink theme out of the box, optimized for e-ink devices
• Added: Added tooltip to API regenerate button to warn user about it invalidating all existing clients.
• Added: Added file parsing support for 'Series 001 (Digital) (Name123)'
• Added: Series detail now shows reading lists where said series has some connection to.
• Added: Series detail will now show Last Read date by the user.
• Added: Implemented the ability to update metadata on series from the UI. Each field can be manually edited and locked. Once locked, no updates will occur on that field, even if the metadata source (ComicInfo) is updated. Fields can be unlocked from the UI. This does not write to the ComicInfo.xml file.
• Added: New, redesigned screens for Chapter/Volume info (now called details). This new modal screen features expanded metadata (from ComicInfo if in underlying file), ability to see the bookmarks in said Chapter/Volume, a quicker way to change cover image, and overall, better experience.
• Added: For archives, when parsing, set the SeriesSort from ComicInfo if it exists (this is a pending v2.1 draft spec)
• Added: Added ETag on headers for Bookmark and Manga images to allow browser to cache some files
• Added: Manga reader now has a modal to show all the shortcut keys
• Added: Manga reader now has a configurable background color in user settings, defaults to black
• Added: Manga reader has a new setting to disable showing screen hints, like overlay when pagination direction is changed. Users can now disable in user settings.
• Added: Cache control on reader images are cached for up to 10 mins
• Added: Added mark as read/actionables on Files in volume detail modal.
• Added: Errors that occur in background tasks, like Backup or Library scan, will now push to events widget to inform the user that said tasks were aborted. User can now click the error and get more information (or check logs for the same information).
• Added: Side nav added to the application. From the side nav you can easily reach your libraries, reading lists, collections, and user settings.
• Added: Hooked in the ability to search by release year (along with series optionally) and series will be returned back.
• Added: Filter screen now lets you filter against a series name, using same contains logic as search. (https://feats.kavitareader.com/posts/93/filter-by-series-title)
• Added: Download role will now block users over ODPS from downloading items. OPDS-PS will still function.
• Added: Added stat collection on new features to help focus development efforts. New stats include: Number of Users on an install, Total Files per Install, Admin's theme, Admin's reading mode, Number of collections and reading lists, and if OPDS is enabled.
• Added: When deleting a library, send Series Removed events for each series in said library, to remove from user's pages without a refresh

Changed

• Changed: Updated to Angular 13 and updated many dependencies
• Changed: Updated to .NET 6.0.2
• Changed: Upgraded to Bootstrap 5
• Changed: Manga Reader will now use regular images to render pages to the user. A canvas (previous implementation) will be used solely for page splitting. This has improved resolution for Fit to Screen rendering on smaller devices.
• Changed: Dark mode is now Custom theme
• Changed: A ton of work behind the scenes to unify the styles we user per component.
• Changed: A major rework to the events widget to provide detailed information about what is occurring during library/series scans and cover refreshes. It will now send filenames being scanned, what series is being updated or where cover generation is.
• Changed: Changed tag badges on the filter to be green instead of outline on dark theme.
• Changed: All toasts relating to task queuing now have 'info' instead of 'success' styles.
• Changed: Width of search bar now spans entirety of navbar on all viewport sizes.
• Changed: Series Detail page is now generated completely in the backend. This allows for greater flexibility and speed. On average, users should see 50% speed increase on page load and site responsiveness.
• Changed: Continue reading or Read button on series detail is now enabled on load. You can skip waiting for the volumes and chapters to load.
• Changed: Series Detail page will now show Books under Books tab only
• Changed: Series Detail page for Book Libraries will now render titles as 'Index - Book title'
• Changed: Chapters/Issues tab will no longer show any chapters that are just a volume (ie Btooom! - v01.cbz)
• Changed: Removed file parsing support for 'A Compendium of Ghosts - 031 - The Third Story_ Part 12' due to complexity in parsing
• Changed: CopyFilesToDirectory will now allow for one duplicate copy over and put (2). This allows for old bookmarks to handle duplicates on download.
• Changed: Removed no-connection route and component since it is no longer in use by the app.
• Changed: Series Detail page now works slightly different. Library type Books will never send chapters back. It will only send Specials or Volumes back. Volumes are renamed to be more user friendly. Non-book types had many fixes to streamline what renders on the screen.
• Changed: Made the badge expander button smaller to match tag badges
• Changed: Log message for Scan Service has been tweaked to make it easier to find total scan time vs scan time for a chunk of work.
• Changed: Multi-select typeaheads now have a close icon to clear out the field
• Changed: Fixed up some styling on Edit Series modal for phones.
• Changed: (Performance) Optimized some code in ArchiveService to avoid extra memory allocations.
• Changed: Tweaked on deck to only show series that have progress within 30 days.
• Changed: Ignore @recycle directories on QNAP devices
• Changed: Disabled some options in the menu when you're in a mode that doesn't support for Manga Reader
• Changed: Moved library type resolution into getChapterInfo resulting in...

v0.5.1.1 - Email Migration Hotfix

There have been a significant amount of reports that users are struggling to receive the email link for email migration or they are unable to find in their logs (usually due to a logging level mismatch). We have altered the migration code to not require any email to be clicked and instead as long as you pass correct login credentials, email migration will successfully validate.

We will continue to look into the email functionality and smooth out existing flows. Thank you for being understanding.

Changed:

• Changed: Email migration will now auto validate the email as long as user provides the correct username/password