-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #13 from tranlyvu/master
Merge master into stable branch
- Loading branch information
Showing
32 changed files
with
378 additions
and
1,154 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,3 @@ | ||
.idea | ||
.idea | ||
env | ||
**.pyc |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,8 +1,12 @@ | ||
language: python | ||
python: | ||
- "2.7" | ||
|
||
- "3.6" | ||
script: python tests/test_all_of_the_units.py | ||
branches: | ||
only: | ||
- master | ||
|
||
install: | ||
- pip install -r requirements.txt | ||
|
||
|
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,26 +1,90 @@ | ||
#finLink: | ||
findLink is a python web-scraping application to minimum number of links between 2 given wiki pages | ||
|
||
[![Build Status](https://travis-ci.org/tranlyvu/find-link.svg?branch=master)](https://travis-ci.org/tranlyvu/find-link) | ||
|
||
#Using findLink: | ||
* Downloading a [release](https://github.com/tranlyvu/find-link) | ||
|
||
* Quickstart | ||
1. Initializing class 'findLink"" | ||
2. Calling "search" method | ||
|
||
* Sample | ||
* [sample](https://github.com/tranlyvu/findLink/sample.py) | ||
|
||
#Contribution | ||
* Join our [gitter] ( gitter.im/find-link) and Maillist | ||
* [Release management](https://launchpad.net/findlink) | ||
* For any feature recommendation please register a [Blueprint](https://blueprints.launchpad.net/findlink) | ||
* For bug reports or requests please submit an [issue](https://github.com/tranlyvu/find-link/issues) | ||
|
||
#Frequently Asked Questions | ||
The FAQs can be found on the project's [Wiki page](https://github.com/tranlyvu/find-link/wiki). Feel free to submit more questions to be answered and added to the page. | ||
|
||
#License | ||
See the LICENSE file for license rights and limitations (Apache License 2.0). | ||
# **wiki-link** | ||
|
||
A web-scraping application to find the minimum number of links between 2 given wiki pages. | ||
|
||
[![Build Status](https://travis-ci.org/tranlyvu/wiki-link.svg?branch=master)](https://travis-ci.org/tranlyvu/wiki-link) [![Code Health](https://landscape.io/github/tranlyvu/wiki-link/master/landscape.svg?style=flat)](https://landscape.io/github/tranlyvu/wiki-link/master) [![Join the chat at https://gitter.im/find-link/Lobby](https://badges.gitter.im/find-link/Lobby.svg)](https://gitter.im/find-link/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) | ||
|
||
<img src="img/link.jpg" width="480" alt="Combined Image" /> | ||
|
||
--- | ||
Usage | ||
--- | ||
|
||
- Set up database management: as of 14 January 2018, Wiki-link has been tested with [Mysql](https://www.mysql.com/downloads/). Alternatively, for quick development setup, I strongly recommend to setup database with [Docker](https://www.docker.com/). | ||
|
||
- Download a [release](https://github.com/tranlyvu/wiki-link/releases) or fork the source code: | ||
|
||
``` | ||
$git clone https://github.com/tranlyvu/wiki-link.git | ||
``` | ||
|
||
- Modify the configuration file for database connection: | ||
|
||
``` | ||
$vi wiki-link/conf.ini | ||
``` | ||
|
||
- The simplest way is to run the [Sample file](https://github.com/tranlyvu/wiki-link/blob/master/sample.py) with your desired wiki links: | ||
|
||
``` | ||
$vi wiki-link/sample.py | ||
$python wiki-link/sample.py | ||
``` | ||
|
||
--- | ||
Development Setup | ||
--- | ||
|
||
Wiki-link was developed using python 3.6, simply run the following on your development environment: | ||
|
||
``` | ||
$pip install -r requirements.txt | ||
``` | ||
|
||
--- | ||
Project Architecture | ||
--- | ||
|
||
To do | ||
|
||
--- | ||
Release History | ||
--- | ||
|
||
* 1.0.1 - Jan 14, 2018 | ||
* Fix database connection bug | ||
|
||
* 1.0.0 - Nov 7, 2016 | ||
* The first official release | ||
|
||
--- | ||
Contribution | ||
--- | ||
|
||
For bug reports or requests please submit an [issue](https://github.com/tranlyvu/wiki-link/issues). | ||
|
||
For new features contribution, please follow the following instruction: | ||
|
||
``` | ||
1. Fork the source code (`$git clone https://github.com/tranlyvu/wiki-link.git`) | ||
2. Create your feature branch (`$git checkout -b new/your-feature`) | ||
3. Commit your changes (`$git commit -am 'Add some new feature'`) | ||
4. Push to the branch (`$git push origin new/your-feature`) | ||
5. Create a new [Pull Request](https://github.com/tranlyvu/wiki-link/pulls) | ||
``` | ||
|
||
--- | ||
Contact | ||
--- | ||
|
||
Feel free to contact me to discuss any issues, questions, or comments. | ||
* Email: [email protected] | ||
* Twitter: [@vutransingapore](https://twitter.com/vutransingapore) | ||
* GitHub: [Tran Ly Vu](https://github.com/tranlyvu) | ||
|
||
--- | ||
License | ||
--- | ||
|
||
See the [LICENSE](https://github.com/tranlyvu/wiki-link/blob/master/LICENSE) file for license rights and limitations (Apache License 2.0). | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
[database] | ||
# Database connection, i.e mysql://root:[email protected]:32771 | ||
connection = mysql://root:13061990@localhost:3306 |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,10 @@ | ||
# The order of packages is significant, because pip processes them in the order | ||
# of appearance. Changing the order has an impact on the overall integration | ||
# process, which may cause wedges in the gate later. | ||
|
||
|
||
beautifulsoup4 | ||
MySQL-python | ||
beautifulsoup4==4.6.0 | ||
bs4==0.0.1 | ||
certifi==2017.4.17 | ||
chardet==3.0.4 | ||
idna==2.5 | ||
mysqlclient==1.3.12 | ||
requests==2.17.3 | ||
SQLAlchemy==1.2.0 | ||
urllib3==1.21.1 | ||
virtualenv==15.0.3 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,9 @@ | ||
from src.findLink import my_findLink | ||
from wikilink.wiki_link import WikiLink | ||
|
||
def main(): | ||
starting_url='/wiki/Barack_Obama' | ||
ending_url='/wiki/Bill_Clinton' | ||
model=my_findLink(starting_url,ending_url) | ||
model.search() | ||
model.printLinks() | ||
starting_url = '/wiki/Barack_Obama' | ||
ending_url = '/wiki/Bill_Clinton' | ||
model = WikiLink(starting_url, ending_url) | ||
print("Smallest number of separation is " + str(model.search())) | ||
|
||
if __name__=="__main__": | ||
main() | ||
if __name__ == "__main__": main() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,27 +1,38 @@ | ||
from setuptools import setup,find_packages | ||
""" | ||
WikiLink | ||
----- | ||
WikiLink is web-scraping application to find minimum number of links between two given wiki pages. It's extensively documented | ||
and follows best practice patterns. | ||
:copyright: (c) 2016 by Tran Ly VU. | ||
:license: Apache License 2.0, see LICENSE for more details. | ||
""" | ||
from setuptools import setup | ||
|
||
|
||
setup( | ||
name='findLink', | ||
description='A web-scraping project to find the links between 2 given wiki pages', | ||
name='WikiLink', | ||
description='A web-scraping application to find the minimum number of links between 2 given wiki pages', | ||
author='Tran Ly Vu', | ||
author_email='[email protected]', | ||
license='MIT', | ||
classifiers=[ | ||
license='Apache License 2.0', | ||
classifiers=[ | ||
'Development Status :: 3 - Alpha', | ||
'Intended Audience :: Science/Research', | ||
'Framework :: IPython' | ||
'Topic :: Scientific/Engineering :: Artificial Intelligence', | ||
'Programming Language :: Python :: 2.7', | ||
'Operating System :: Microsoft :: Windows :: Windows 7', | ||
'Programming Language :: Python :: 3.6', | ||
'Operating System :: Microsoft :: Windows :: Windows 10', | ||
'Natural Language :: English', | ||
'License :: OSI Approved :: MIT License', | ||
'License :: OSI Approved :: Apache License 2.0', | ||
], | ||
keywords='web-scraping', | ||
packages=['findLink'], | ||
packages=['wiki-link'], | ||
install_requires=[ | ||
'beautifulsoup4' | ||
'bs4', | ||
'MySQLdb' | ||
'urllib2' | ||
'SQLAlchemy' | ||
'requests' | ||
] | ||
|
||
) |
This file was deleted.
Oops, something went wrong.
Binary file not shown.
Oops, something went wrong.