Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for saving user action logs #189

Open
ftyers opened this issue Dec 9, 2017 · 9 comments
Open

add support for saving user action logs #189

ftyers opened this issue Dec 9, 2017 · 9 comments

Comments

@ftyers
Copy link
Collaborator

ftyers commented Dec 9, 2017

It would be cool to keep a track of how the user is interacting with annotatrix, this could provide interesting data for training models and for investigating how annotators annotate. In the simplest way
you could have an entry for e.g. AnnoTime=1512811353 (unixtime) in each token in the MISC column. Edits could be represented simply by comma separated values, AnnoTime=1512811353,1512811427.

Additional thoughts and suggestions welcome!

@jonorthwash
Copy link
Owner

Do you think this logging should be done client-side, or server-side (i.e., only when using server mode)?

@ftyers
Copy link
Collaborator Author

ftyers commented Jan 5, 2018

If there is a server it can log more stuff, if it's user side it should only log the last change unixtime in the MISC column. I don't think it's a good idea to clog the misc column, but i think having annotime could be an interesting feature, and easy to store.

@jonorthwash
Copy link
Owner

Oh! I somehow missed that you had in mind keeping that sort of metadata in the conllu. Last edited time certainly could work that way, but I'd be worried about accumulating too much stuff of that sort, depending on what else you have in mind.

@ftyers
Copy link
Collaborator Author

ftyers commented Jan 6, 2018

I think in the CoNLL-U it should be completely restricted to just AnnoTime. In the server logging we could have more detailed info.

@jonorthwash
Copy link
Owner

Agreed. What sorts of things would you have logged by the server?

@ftyers
Copy link
Collaborator Author

ftyers commented Jan 7, 2018

Well, I have been thinking about stuff that might be useful to training algorithms... for example:

  • Which thing was annotated first
  • If relations are changed x was changed to y
  • How long it takes to annotate a sentence

I'm sure there is more stuff too.

@gstark0
Copy link
Contributor

gstark0 commented Jan 7, 2018

@ftyers So, at the client-side, it should add AnnoTime entry to misc column, when token was created, and then, after comma, log every change to that token? Including changing name, creating deprel, making supertoken, splitting token?

@ftyers
Copy link
Collaborator Author

ftyers commented Jan 7, 2018

@gstark0 for now I would just add support for updating the AnnoTime for the last changed time later we can decide if we want to log a timestamp every change (it could be a lot of timestamps).

@gstark0
Copy link
Contributor

gstark0 commented Jan 9, 2018

@ftyers Is this how it should look like -> #209 ? It's just the beginning, I would like to improve it, but I want to make sure that its a good start.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants