Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Alessandro Melchiorre
last_fm_crawler
Commits
c30f085c
Commit
c30f085c
authored
Jun 18, 2020
by
Alessandro Melchiorre
Browse files
# Added prints
parent
8a092360
Changes
1
Hide whitespace changes
Inline
Side-by-side
src/main.py
View file @
c30f085c
...
...
@@ -40,7 +40,7 @@ lfm1b = lfm1b.reset_index(drop=True) # Generates new index
lfm1b
=
lfm1b
[
lfm1b
.
index
%
number_of_crawlers
==
this_crawler
]
print
(
lfm1b
.
head
())
print
(
"Number of users
to crawl
is: {}"
.
format
(
len
(
lfm1b
)))
print
(
"Number of users
in the split
is: {}"
.
format
(
len
(
lfm1b
)))
# timestamp for 20/03/2020 12:00:00 GMT
to_ts
=
1584705600
...
...
@@ -78,6 +78,7 @@ if not work_on_failures:
print
(
"Number of users failed is: {}"
.
format
(
len
(
failed_users
)))
lfm1b
=
lfm1b
[
~
lfm1b
.
username
.
isin
(
failed_users
)]
print
(
"Number of users to crawl is: {}"
.
format
(
len
(
lfm1b
)))
# api_key_idx is included in crawling_settings
crawler
=
LastFMCrawler
(
api_key
=
API_KEYS
[
api_key_idx
],
api_secret
=
API_SECRETS
[
api_key_idx
],
to_ts
=
to_ts
)
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment