Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for playlist management (v2) #287

Open
wants to merge 34 commits into
base: main
Choose a base branch
from

Conversation

girst
Copy link
Member

@girst girst commented Nov 13, 2020

This is now done :D

I've adapted Myers' diffing algorithm (the one that's used by e.g. git-diff) to support moving ranges of tracks. It generates a relatively optimal edit script, to minimize API requests without the diffing algorithm taking too long. Especially getting moves to work was quite tricky (took me over a week of daily head-scratching), but I have written a whole bunch of tests for it. the web_client_mock now also supports editing playlists.

One thing we haven't yet talked about is what to do when API requests fail. The worst case (and only serious one) is when we clear-and-repopulate a playlist. When we fail mid-way through, we lose data. Should we do something (e.g. save an m3u playlist somewhere) then?

What's still todo is:

CC: @blacklight
closes #22, closes #236

blacklight and others added 7 commits November 3, 2020 19:34
- can't pass in sets, only lists
- provide a useful default n parameter
making playlist private, not just for the obvious reason, but also
because our API client is not authorized for the playlist-modify-public
scope (only playlist-modify-private).
@codecov
Copy link

codecov bot commented Nov 13, 2020

Codecov Report

Merging #287 (1c0358a) into master (66d40fc) will increase coverage by 0.42%.
The diff coverage is 97.77%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #287      +/-   ##
==========================================
+ Coverage   96.68%   97.10%   +0.42%     
==========================================
  Files          13       13              
  Lines        1207     1418     +211     
==========================================
+ Hits         1167     1377     +210     
- Misses         40       41       +1     
Impacted Files Coverage Δ
mopidy_spotify/browse.py 91.30% <ø> (ø)
mopidy_spotify/playlists.py 97.22% <95.68%> (+2.93%) ⬆️
mopidy_spotify/translator.py 96.35% <100.00%> (+0.01%) ⬆️
mopidy_spotify/utils.py 100.00% <100.00%> (ø)
mopidy_spotify/web.py 96.63% <100.00%> (+0.54%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66d40fc...1c0358a. Read the comment docs.

Comment on lines 318 to 321
# Note: date can by YYYY-MM-DD, YYYY-MM or YYYY.
date = web_album.get('release_date', '').split('-')[0] or None

return models.Album(uri=ref.uri, name=ref.name, artists=artists, date=date)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to extend the track translator to parse release dates (just the bare minimum (year); spotify also supports finer dates) to support round-tripping between the web and native formats. otherwise, I wouldn't have been able to compare output from save() and lookup().

@girst girst force-pushed the bug22 branch 2 times, most recently from 0f18224 to 6c5d9a0 Compare November 15, 2020 14:45
@girst
Copy link
Member Author

girst commented Nov 15, 2020

I found and fixed the problem with the cache (the playlist's tracks are fetched from two endpoints, not just the one I cleared). While at it, I also moved the newly added code to use the new shorter playlist URIs that spotify offers (as the old ones are deprecated). Also started testing this out on my personal spotify account, and it pretty much works.

This leaves us with only one major item to figure out: what do do on errors. For this, I'd like to solicit some ideas from you guys.

Uses Myers' diff, extended to detect moved ranges. With the exception of
non-consecutive deletes, this generates a near-optimal edit script.
To use the preexisting mopidy_album_mock the web translator had to be
extended to parse release dates (only the year is supported, as that's
what the test data requires).
@girst
Copy link
Member Author

girst commented Nov 20, 2020

OK, so I think this is done. I'm going to add @jodal, @adamcik and @kingosticks as potential reviewers. Looking forward to your feedback!

PS: in case you feel initmidated by the patch size: 70% of it is just tests.

edit nov21: just for fun, i did some calculations (on commit 468a811): only 230 lines or 19% of this patch is actual non-test code (excluding comments, closing braces and blank lines):

signontest=$(git diff upstream/master bug22 mopidy_spotify/ | grep -v '^diff\|^index\|^+++\|^---\|^@@\|^ ' | cut -b2- | grep -v '^ *$' | grep -v '^ *#' | grep '^ *\S\{3\}' | wc -l)
allchanges=$(git diff upstream/master bug22 | grep -v '^diff\|^index\|^+++\|^---\|^@@\|^ ' | wc -l)
echo significant non-test changes: $(( 100*signontest / allchanges ))'% (' $signontest lines ')'

# significant non-test changes: 19% (230 lines)

@girst girst requested a review from kingosticks November 20, 2020 13:19
@girst girst force-pushed the bug22 branch 2 times, most recently from 52e64c4 to c34a7b0 Compare January 2, 2021 22:37
@kingosticks
Copy link
Member

There are quite a lot of tests verifying the correctness of the diff calculation (and every one of them is testing some other potential bug); generating Responses for each of them seems very tedious. and given that the main point in those tests is verifying that the diffing works, won't be very useful either.

I agree, responses at this level would be unpleasant and also not useful. But testing multiple things isn't ideal. I think we can restructure a little and test at lower levels to avoid writing too many Responses and hopefully also avoid a Python version of the Web API. It may end up being more boilerplate but I'll have a go and see where I land. Either way, doing this will probably help me review.

mopidy_spotify/utils.py Outdated Show resolved Hide resolved
@kingosticks
Copy link
Member

Sorry, I did a few bits, hit a small issue, managed to context switch along the way and lost my momentum. Would you mind rebasing on master when you get a spare moment.

@girst
Copy link
Member Author

girst commented Jan 10, 2021 via email

@kingosticks
Copy link
Member

I've got another review in progress but it's not quite done yet. Probably less work for you if it's all at once.

@blacklight
Copy link

Hey guys, what's the current status with this PR?

@kingosticks
Copy link
Member

It's not yet merged. It's waiting on me, it's my fault. I've been meaning to dig this out for weeks and I have only a couple of days before I'm unavailable for a while. Nothing motivates like an impending deadline.

@girst
Copy link
Member Author

girst commented May 4, 2021 via email

@kingosticks
Copy link
Member

kingosticks commented May 4, 2021

We've all spent a lot of time on this and it deserves to be finished; I think in your shoes I might have been quite annoyed so I mainly want to say I really appreciate the patience and understanding here. I did lose momentum and focus on this, and have also scaled back how much time I invest in this hobby but this kind response is exactly the motivation I need. Having said that, I'm about to take on an enormous priority shift (complete with a couple of weeks off work and zero free time) so while I would love to say I'll properly get back into this right now but I'm not sure that's remotely feasible for a few weeks.

@dsynkd
Copy link

dsynkd commented Mar 22, 2022

@kingosticks any chance we could get this approved?

@kingosticks
Copy link
Member

I spent time reviewing this and reworking the tests to fit the way we had done things previously - I don't want to maintain the code as it is. I have that local branch somewhere, I have been meaning to pick it up for the last 10 months I just have not had the time to invest and get back into it. So yes, there is a chance to merge this in a slightly different form. I should publish that branch and then someone can finish it off rather than wait any longer.

@girst
Copy link
Member Author

girst commented Mar 24, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Spotify playlist management
4 participants