Status Reports
All features described on the features page have been implemented and deployed to the production site, excluding any specifically marked as removed or future features. The project is in line with the roadmap laid out by the team during the spring and summer.
Andre will continue to support the Wikiframe environments while working as a volunteer. His work will focus on break/fix issues, server monitoring, and planning/presentations about Wikiframe. Hours won't exceed five per week.
Now that Wikiframe is available outside the firewall, Johnny, John, and I will pursue some tasks to ensure the system is well-supported going forward. Here's a list of known tasks:
System/Environments
- Custom error pages for the web server. These will notify users what's likely going on, and what to do when they get a "404" or "500" error.
- Review existing server security and config setup for any needed changes.
- Create multiple Wikiframe server environments: test, fallback, and production.
- Move Git repository to unlv server.
- Write "runbook" for deploying, configuring, and troubleshooting the app.
- Get the mysqlimport utility working on the server: for updates to Wikiframe config/SPARQL tables.
- Get Django administration app pages working in server environments - for ongoing maintenance of config data in Prod.
New Features
- geographic map views
- utility for scheduling jobs: Wikidata caching and error.log cycling (in progress)
- automatic updates to browser caches when static files change.
- others TBD
Work done this week
- faceted search completion & deployment (dev 17)
- analysis of error log noise from external bots
- work on custom error pages (Django-specific solution forthcoming)
Work for next week
- create test/fallback servers (depending on J & J's availability)
- custom error page deployment
- troubleshoot/test scheduler
- geo maps research and dev
Work this week
- queue debugging
- other search engine debugging
- standardization and refinement of terms used in source code (code maintenance issue)
Work for next week
- Andre to write test scripts; Darnelle & Cory to review
- Deploy dev.13 to wikiframe server, with help from John & Johnny
- multi-user testing start-up (hopefully)
- on-going defect resolution from testing
Dev.12 Features
- updated page headings to reflect name change, etc.
- query history selection box; keeps prior three queries, by facet.
- "back to results" button on item details page.
- hyperlinks for ark URIs and other properties containing "http..." values.
- status message and spinning gif while graph loads (a bit hokey, I'm working on a styled progress bar to replace it).
- facet tabs stay highlighted after user selects.
- NOTE: The new deployed version is acting buggy. Probably a glitch in the current deployment process. Will address on Monday.
Next Week:
- More work on UX functionality.
- More class-based view refactoring.
- Sprint demo on Friday.
Work Completed Last Week:
- Johnny renamed app server to "wikiframe.library.unlv.edu"; working on obtaining valid SSL cert.
- Darnelle reviewed application version dev.10.2 and provided change requests to Andre. See the project page for details.
- Andre added all current SPARQL source code to the git repository.
- Andre made changes from Darnelle's list: branding & messaging; status animation while graph is loading.
- Andre began work on consolidating four-page results functionality into a one-page results class.
Sprint Plan
- Continue one-page results redesign.
- Continue making Darnelle's requested changes.
- Deploy dev.11 to wikiframe.
- Had to overcome MySQL security issues following Drupal installation (should have done more research!). Spent two days recreating a clean MySQL Server installation after unintentionally installing MariaDB as part of the Drupal installation.
Worked Completed:
- Resolution of numerous issues around MySQL installation and authentication.
- rebuild and installation of mysqlclient module needed for the application.
- completed defect resolution for querying list and graph for all domains (people, corps, collections, oral histories).
Next Week:
- Deploy application dev.8 to ore.library.unlv.edu
- work on Knowledge Panel and Home Page features.
Headlines: Graph functionality updated to support multiple relationship types; Code refactored, defect resolution in progress.
Work Completed:
- Multi-select of graph relationship-type.
- All graph nodes show as ellipses.
- Switched node type evaluation from shape to color.
- Additional code refactoring
- Sprint 3 demo.
Work Plan 5/1 - 5/12
- Complete post-refactor debug. Adding multiple relationship types by domain created more paths through the code. Will take a little more time to stabilize.
- Do initial deployment to ore.library.unlv.edu. Should be ready to deploy by mid-week.
- Add selected hyperlinking of properties.
- start work on knowledge graph.
- Capture requirements for data downloads.
Work Completed:
- application vm set up and reviewed. User accounts and permissions set up for Ubuntu, MySQL, Apache, and the /data/www/ directory, which will house the application installation. Initial installation planned for week of May 1.
- Presentation of randomly selected images for People and Corporate Bodies landing pages (see screenshot below).
- Refactored views layer to optimize form processing.
- Fully integrated Corporate Bodies search & graph.
- started graphing relation multi-select feature (FR003c).
- removed tooltip entries with missing values (FR003g).
Next Week
- Complete multi-select of graph relationship-type.
- Hyperlink selected property values on pages.
- Show all graph nodes as ellipses.
- Switch node type evaluation from shape to color.
- Sprint 3 demo (Friday).
Headlines: Graph shows entity-specific node properties in hovering tooltip for Persons and Collections; New Knowledge Panel feature; Updated Home Page feature.
Work Completed:
- Switched application database from PostGRES to MySQL 8.
- Refactored some data access code and wrote functions to replace one Django querying feature not supported for MySQL (I've requested the Captains of Django to address the issue!)
- Data integration and presentation for the Oral Histories tab.
- Removed data loading when user initially clicks a tab. All data comes courtesy of searching. Accepted by Darnelle and Cory at sprint demo.
- User now able to dynamically search by graph node selection.
Sprint 3 Plan:
- Review of application vm set up by John.
- refactor some code in the Views layer and improve code commenting.
- Add Corporate Bodies feature
- pick relation-type feature
- hyperlink selected property values on pages.
- Start Knowledge Panel feature (hopefully).
Headlines": Graph visualizer updates dynamically according to search; user can filter collection results by one or more selected subjects.
Worked Completed:
- Visualization library locally installed and integrated with project files.
- Graph updates dynamically based on search results for people. - see image.
- basic toolip feature when hovering over graph node. - see image.
- hook built for initiating search result/graph update based on graph node selection - see image.
- Application caches subjects in database.
- User can search subjects, select one or more results, and filter collections. -- see image.
- Meeting with sys admins on 4/6: Andre planning for start of user testing in early May; John & Johnny to create Ubuntu 22.04 virtual server with MySQL 8.1, Django 4.04, Python 3.10, Apache (ver?); Jonathan to send VPN set up instructions to Andre; Andre to validate switch from PostGRES to MySQL backend.
Work for next week:
- Graph Visualizer: styling for tooltip on node hover; finish dynamic querying through selected graph nodes.
- testing MySQL database technology for application backend, per sys admin request.
- Andre to set up VPN connection.
- Corporate Bodies: Create application objects and user interface presentation.
- Oral Histories: Create application objects and user interface presentation.
- This may require multiple days of work.
Sprint 2 Plan:
- Graph Visualizer: import external Vis.Js library into local project; record library version. Integrate graph view with filtering process so that graph updates dynamically after a keyword search. Integrate graph relationship pick list so users can set one or more properties for the graph.
- Corporate Bodies: Create application objects and user interface presentation.
- Oral Histories: Create application objects and user interface presentation.
- Subjects: -- Formerly 'Topics' -- Update query used to retrieve master list; create filter functionality in which select subject(s) return collections and/or oral histories having the subject(s).
Work this week:
- Vis.js integration and optimization.
- basic presentation for Corporate Bodies and Oral Histories.
-
Install the PostGRES database and administrative tools: this is the database the app will use when it's publicly available. It's open source and thus free, and I have some familiarity with it.
-
Create a Django project for the app: this included Filter record
- configuring it to work with PostGRES
- configuring base site files and their locations
- making a local Git code repository
- integrating the SPARQLWrapper library
- writing baseline code to run Darnelle's people query and cache it in in the database
- configuring and launching the admin subsite that comes with Django
- Work in progress:
- creating the baseline templates that present the web pages. Not much coding inside these yet; just enough to see them work.
I inserted a custom tag in the SPARQL that works like a filter (see below). When the query is called, the app inserts the qcode into the query so that the scope can be changed to a different institution w/o rewriting the app.
Here's the query, now stored in the database and accessible via the admin app. We can drop as many saved queries as we like into this table and access/run them from inside the web app. SPARQL record image