Thomas Wade
490a2ac22d
Add dividers and some missing comments
2018-09-25 19:44:52 +09:30
Thomas Wade
829374fc6e
Add some configuration options near start of script
2018-09-25 19:37:55 +09:30
Thomas Wade
295e3e3d12
Track users and tracks in a database-like fashion
...
Rather than have several copies of a user/track object, collect them all in a single dictionary and give the keys to each user object. This will avoid duplication and make generating graphs easier.
2018-09-25 19:09:05 +09:30
Thomas Wade
6e5cfdfa0d
url should be None by default
2018-09-25 19:04:44 +09:30
Thomas Wade
7de170efc9
Mark users as processed and check when printing
2018-09-25 19:03:44 +09:30
Thomas Wade
0c496ecd71
Add constructor for User class
...
This was causing issues where appending to any of the lists would update the class variable rather than the instance variable, leading to everyone having the same likes, tracks, etc.
2018-09-25 19:01:07 +09:30
Thomas Wade
9e9a56386f
Organise tracks by url rather than artist-author mishmash
2018-09-25 17:23:53 +09:30
Thomas Wade
fa9517a21d
Remove track dict that was inside track class
...
wtf? why was this here?
2018-09-25 17:22:55 +09:30
Thomas Wade
ae3e32cd21
Get track URLs
2018-09-25 17:22:28 +09:30
Thomas Wade
2e06dc231f
Enforce limits
2018-09-25 17:21:23 +09:30
Thomas Wade
8ef89e373b
Process follow(ers/ing) of each user
2018-09-25 16:55:43 +09:30
Thomas Wade
fc1b2ca4cd
WIP: adding differentiation between ego distance
2018-09-24 13:11:21 +09:30
Thomas Wade
d6dbc472bd
Scrape users' follower and following lists
2018-09-24 12:58:01 +09:30
Thomas Wade
ca49b6d1b6
Get users' likes
2018-09-20 01:00:15 +09:30
Thomas Wade
539580de2a
Ignore collections
...
I really don't want to iterate over collections so we'll just ignore them outright
2018-09-20 00:59:39 +09:30
Thomas Wade
cdf78f0208
Sprinkle a bit of NoneType safety into the mix
2018-09-20 00:59:13 +09:30
Thomas Wade
cde49030dc
Use a list to store users' tracks and likes
2018-09-20 00:58:48 +09:30
Thomas Wade
aff01870b5
Scroll until limit is reached
2018-09-20 00:54:30 +09:30
Thomas Wade
3decee9b03
List tracks
2018-09-19 20:49:54 +09:30
Thomas Wade
8b4c3f5489
Genericise Get_Tracks_Info to take any infiniscrolling list URL
2018-09-19 20:48:21 +09:30
Thomas Wade
33dd9edcdc
Actually use element provided by loop
...
*facedesk*
2018-09-19 18:37:04 +09:30
Thomas Wade
8c8631df7c
Scroll through tracks page
2018-09-19 15:11:22 +09:30
Thomas Wade
5781fab7bd
Add some better structure to status prints
2018-09-19 09:49:01 +09:30
Thomas Wade
3f8aa7304f
Scrape tracks
...
Some scrolling is still needed to actually get the whole track list
2018-09-18 20:01:38 +09:30
Thomas Wade
30eebeef90
Extract basic info scraping into function
2018-09-18 18:35:57 +09:30
Thomas Wade
e5b2e8861a
Print what's happening while scraping
2018-09-18 18:21:30 +09:30
Thomas Wade
4f36369d84
Get follower and following count
2018-09-18 18:13:07 +09:30
Thomas Wade
51624e9715
Use user object where possible
2018-09-18 18:08:13 +09:30
Thomas Wade
9b9345887b
Check if element exists before parsing
2018-09-18 17:45:54 +09:30
Thomas Wade
b8a6a431bf
Initialise to None
2018-09-18 17:45:01 +09:30
Thomas Wade
0846626460
Print out collected details of each user
2018-09-18 17:39:42 +09:30
Thomas Wade
e94c4ac596
Add Bossfight to users_to_process
2018-09-18 17:39:16 +09:30
Thomas Wade
0eedc67144
Use User class instead of dictionary
2018-09-18 17:38:48 +09:30
Thomas Wade
7792e5cbda
Correct copy-paste
2018-09-18 16:35:43 +09:30
Thomas Wade
3879d437e2
Store collected data in user_dict
2018-09-18 15:54:37 +09:30
Thomas Wade
c45391c436
Move comment for consistency
2018-09-17 23:58:49 +09:30
Thomas Wade
7cb2e2a2ff
Get username, track count, and like count
2018-09-17 23:55:53 +09:30
Thomas Wade
7c7c016e27
Disable image loading
2018-09-17 23:55:18 +09:30
Thomas Wade
65c92026d9
Add initialisation messages
2018-09-17 23:55:01 +09:30
Thomas Wade
8241702fb1
Don't get page in setup phase
2018-09-17 23:54:06 +09:30
Thomas Wade
6ca88dfd94
Capitalise ChromeDriver
2018-09-17 23:52:21 +09:30
Thomas Wade
8faf429e0c
Initial commit
...
So far just pulls out the name of the target user
2018-09-17 19:06:39 +09:30
