Home

Welcome to the Roboragi wiki! There's not a lot here yet.

This will be cleaned up and moved into a project soon.

To-do:

• Do before any serious work! Change the license! Affero GPL looks most appropriate. Goals = keep publicly usable derivative works open source
• Actually follow a standard with regard to variable names, function names and formatting
• Improve code reuse
• Make comment formatting less shit. No more building them up on the fly as a string
• Update existing documentation here and on Reddit
  • Add autodoc? https://readthedocs.org/?
  • Add autodoc for the Reddit wiki as well? Especially useful for synonyms.
• Change PostgreSQL to something lightweight and portable (something like a document-based SQLite would be ideal - no need for a relational DB)
• Add proper logging with meaningful data
  • On the exception side - stack track, time, comment id if relevant
  • On the Reddit side - comment id, time, what the comment being processed was and what was found by Roboragi
• Add the ability for users to send a PM and delete or edit a comment of Roboragi's
• Actually have unit/E2E tests
  • Need to make it easy to tell when the HTML-scraped data sources change and stuff starts failing
  • Integrate CI with Github
  • Once there's adequate test coverage, look into CD
• Standardise the "data source" code where possible - need to make it easier to add/extend new anime/manga databases
• Change MAL's data source to use Jikan instead, then re-add them as a primary data source
• Change all data sources to return a common object - will make it much easier to format comments (just grab a bunch of the common object and "flatten")
• Store all entries returned by the data sources and build up a "cache" database for faster response times/reduced strain on data sources/better resiliency if a source goes down (looking at you MAL)
  • Requires "common objects" as listed above
  • Will require a rework of the DBs and the way Roboragi searches - "cached" data can be updated infrequently (once a month for most entries, once a week for ongoing entries - either scheduled or on demand)
  • Add double metaphone searching to help with spelling mistakes? Requires cached data for searchability - existing data sources won't have the ability
  • Change the way manual synonyms/overrides are added/managed - way too much work currently, would be easier to just add a single word to a "master" database
    • Even better would be a way to have users suggest stuff themselves - maybe a website?
• Mods being able to add themselves to the "active subreddits" list automatically would be neat.
• Website with live stats would be nice - maybe expose Roboragi as an API?
• Weekly stats posts (trending, on the rise, lowering etc). Most recent stats link should be in signature
• Proper documentation would be nice
• Having everything spun up and running in Docker would be super cool - need to make it easier/cheaper to deploy/run - cloud-hosted VM not economically viable long-term.
  • Would be nice for other people wanting to learn from Roboragi - too hard to spin up casually atm
• Would be nice to bundle the "search" code as a Python library for others to use.
• Add the ability for Roboragi to process images on request and respond with what he thinks it is