Home

Welcome to the big-data wiki!

Discussion of each group members experiences and expertise

• Izaak - Everything.
• Eric - stuff
• Jihoon - C++, C#, JAVA, Python, and some working knowledge with Web Front-end (ActionScript, JavaScript, CSS and JQuery)
• Mark - Web Front-end (JavaScript, JQeury, JQuery Mobile, CSS)

Splitting each group members roles into the following tasks

• CSS layout design - Izaak
• PKI Setup - Izaak
• Coverflow Javascript - Mark & Jihoon
• Infinite Flickr Coverflow Photos - Mark
• Infinite scroll Javascript - Jihoon
• Fetching Flickr Data Javascript - Eric

Any challenges we have confront with:

• Public Key Infrastructure (PKI) Security -> Izaak

Questions to answer

(i) What are the demonstrated advantages of the technology you are developing?

• Targeted search because you have access to specific facets of the object
• Better visual representation that aids user with the contents of different sources * layout is customizable
• Organization of wide variety of different sources into one unified management system * standardize information * organized collection of data
• Mobile: not limited to platform or device * display of information on a mobile device is customizable (css)
• Category specific results vs search engines (not needed): * Searched results are determined by the specific category selected by the user * information specific to the search is delivered to the user * eliminates the need to visit each individual source (links) * faster consumption of information
• Results from search engines are generally broad and produce irrelevant links to web pages (not needed) * requires ‘sifting through dirt to find the gems’

(ii) What are the risks and limitations of the technology you are developing?

• Finding all necessary API’s for different sources
• API’s gets out-of-date or is longer supported
• Current prototype only works in Google Chrome (browser dependent)
• Security (need access to SFU’s central servers)
• Legal/copyright issues (replication of information)
• Through website testing (authentication, data retrieve,
• Responsiveness vs native app
• Limited storage and display on mobile * information would have to be limited or reduced to be presentable
• Users might not like the 'drill down' method of searching or cannot effectively use filters * choice overload * does exploratory search work well? (users are unsure about what they want to search) * a lot of navigation - (solved by ES?)
• Data to display? * allow users to choose attributes to display

(iii) What bounds have you been able to place on the extent of the risks of the technology? What are the most worrisome risks? What plans do you have for narrowing the bounds on those risks?

Worrisome Risks:

• Security Risks (making sure we get access to SFU’s central server)
• Web development learning curve
• Source code management
• Testing the website
• Wrong time estimation, progress could not meet the schedule
• All software resources may change without any notification during the period of testing

** Plans:

• If accessing to SFU’s central server is not possible, we could generate our own mockup data to retrieve
• Work closely with each other and pair a group with one who is more experienced than other
• Use Github to manage our code so that everyone has up-to-date working code
• Test scripts, test environment and dependencies should be addressed during test meeting
• Overtime work may be required for all the team members
• Test group will address any testing related issues to the team
• Come up with possible assumptions and resources constraints in earlier stage so that everyone is aware of the project limitation

--------------------------- Meeting @ May 22, 2012 ** Current Sources of Data:

• Three Sources (> 77, 000):
• Consumption of this data -> Elastic Search to do basic query and how to access them quickly ** Views for data
• Infinite scroll list, coverflow
• Prefetch data on the fly ** Deployment Server
• Push code from GIT ** Wiki on GIThub

Risks:

• Learning Curve (GIT, Node JS, Javascript, HTML5, CSS3, HTTP, Elastic search, JQuery)
• Performance
• Latency
• Data Ownership
• Responsive Ui

Managed Risks:

• Deployment

Risk - Chance or possibility of loss Regret - amount of loss a person can tolerate Reward - Risk = hazard x uncertainty

Likelihood of occurence | Impact if it occurs | Mitigation | Contingency | Trigger (first)

@ May 22, 2012 TO DO:

• import/export of data
• dump index > output.json (stdout)
• import index < output.json (stdin)
• filter (elastic search demo page)

[object properties] [operator (+, -, >, <)] [value] + - ==> into ES Query

song ex) artist equals Bob

@ May 29, 2012

Done:

• import/export tool

• DB sources

• 120k records

• UI improvement

• coverflow => views

• grid/list => views

• query builder

• searches

• filters

To-Do:

• mobile support
• new data sources
• putting it all together(integration)
• documentation
• build queries based on search
• updating source