Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload Non-CR3 file #1641

Open
wants to merge 24 commits into
base: john/20316-nested-object-support
Choose a base branch
from

Conversation

johnclary
Copy link
Member

@johnclary johnclary commented Dec 26, 2024

Associated issues

This PR adds the ability to upload a CSV of non-CR3 crashes.

Screenshot 2024-12-26 at 2 56 33 PM

Testing

URL to test: Local

  1. Use the new sidebar item to navigate to ./upload-non-cr3
  2. Use the Download CSV Template link to download the CSV template
  3. Try importing the template you downloaded - it has errors!
  4. Correct your template by deleting the problematic rows—or fixing them.
  5. Upload the valid CSV.
  6. Modify your CSV file by duplicating one or more rows. Try to upload it, and verify that it fails to pass validation.
  7. Bonus points: use your SQL client to verify your CSV records have been created in the db.

I think that's it!


Ship list

  • Check migrations for any conflicts with latest migrations in main branch
  • Confirm Hasura role permissions for necessary access
  • Code reviewed
  • Product manager approved

@johnclary johnclary changed the base branch from main to john/20316-nested-object-support December 26, 2024 17:56
@johnclary johnclary changed the base branch from john/20316-nested-object-support to john/19979-location-crashes December 26, 2024 17:58
@johnclary johnclary changed the base branch from john/19979-location-crashes to john/20316-nested-object-support December 26, 2024 17:58
@johnclary johnclary changed the base branch from john/20316-nested-object-support to nextjs December 26, 2024 18:00
@johnclary johnclary changed the base branch from nextjs to john/20316-nested-object-support December 26, 2024 18:00
@johnclary johnclary added the WIP Work in progress label Dec 26, 2024

const onSelectFile = useCallback(
(file: File) => {
Papa.parse<NonCr3Upload>(file, {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the old version of this feature used react-csv-reader, which is just a wrapper around papaparse.

</AlignedLabel>
</Alert>
)}
{validationErrors && (
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the old version of this feature used a fancy table component that enabled the user to edit the CSV and re-submit it through the UI.

I opted to keep things simpler. AFAIK Xavier uses this feature four times per year.

/**
* This is a 4x4 decimal degree bounding box centered on the tx state
* capitol building
* */
Copy link
Member Author

@johnclary johnclary Dec 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copied from the old VZE

@@ -0,0 +1,86 @@
import { z } from "zod";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Zod is back! and it's making quick work of a bunch of validation code we used in the old editor.

*/
export const NonCr3UploadDedupedSchema = z
.array(NonCr3UploadSchema)
.refine(noDuplicateCaseIds, "Duplicate case IDs (case_id) detected");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how common this problem is, but it was in the old VZE so i went ahead and added the test here. Without this check, Hasura is going to bomb with a very unhelpful error.

Copy link
Contributor

@mddilley mddilley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this new iteration, and I think it covers all the bases. 🙌 🚀 I requested changes because the template file is missing from the diff so it is a small fix. I also had one question about the validation behavior.

type="file"
name="file"
accept=".csv"
onChange={(e: React.ChangeEvent<HTMLInputElement>) => {
Copy link
Contributor

@mddilley mddilley Dec 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i tested the invalid template, and, when I removed the problem rows from my file and tried again without refreshing the page, I'm not seeing this onChange function fire again to clear the errors. I might need someone else to test as well or confirmation that this is expected.

The errors do clear when I refresh or navigate away and back to retry the same file.

</Col>
<Col className="my-auto">
<Link
href="/files/non_cr3_template.csv"
Copy link
Contributor

@mddilley mddilley Dec 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johnclary this file is missing from the public folder but I was able to test with the template from staging. 🥼

edit: I just realized that .csvs are gitignored at the root project level so that makes sense why it snuck out of this diff. I was so confused when I wasn't seeing a git change locally when i added the file to my local project. 😵‍💫

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants