yt-videos-list

• BUGFIX

  • fix pip installation problem due to incorrectly formatted
    version specifiers
  • update video duration extraction to correctly
    extract the duration of each video and avoid
    writing 'N/A'

• FEATURE IMPROVEMENTS

  • improve identification of seen videos in csv files by
    • avoiding potentially brittle regular expression matching
    • parsing each row of the csv file and extracting the
      (Video ID|Video URL) value from the corresponding column directly
  • normalize whitespace to avoid including newlines,
    carriage returns, and multiple consecutive whitespace characters
    in the video title
  • improve logging messages by including time.time() and
    time.perf_counter() when logging the time taken to perform
    an operation

• PERFORMANCE IMPROVEMENTS

  • increase write efficiency by completely avoiding writing to a
    temporary file when no new videos found for an existing file

• INTERNAL IMPROVEMENT

  • the following change does not affect the functionality of the program
    • add unit tests for the video title whitespace normalization

• BUGFIX
  • around mid-late October, YouTube rolled out a new UI that changed
    rendering of different parts of the website, including the videos
    page
  • this broke the previous scraping logic, and this release fixes
    the endpoints to correctly extract video information
  • for more information, see the following references:
    • https://techcrunch.com/2022/10/24/youtube-rolls-out-new-design-with-pinch-to-zoom-on-ios-and-android-and-other-updates/
    • https://blog.youtube/news-and-events/an-updated-look-and-feel-for-youtube/
    • https://www.youtube.com/watch?v=zC8hYQwiEy4
    • https://www.youtube.com/watch?v=lYVtgYjJvuo

• BINARY UPDATES
  • Mozilla Firefox
    • geckodriver v0.32.0 (Firefox versions ≥ 104)
    • geckodriver v0.31.0 (Firefox versions ≥ 99)
  • Opera Stable 82, 83, 84, 85, 88, 89, 90, 91, 92 & 93
    • operadriver v.107.0.5304.88 (Opera Stable 93)
    • operadriver v.106.0.5249.119 (Opera Stable 92)
    • operadriver v.105.0.5195.102 (Opera Stable 91)
    • operadriver v.104.0.5112.81 (Opera Stable 90)
    • operadriver v.103.0.5060.66 (Opera Stable 89)
    • operadriver v.102.0.5005.61 (Opera Stable 88)
    • there was no operadriver release specifically for version 101 (Opera Stable 87)
    • there was no operadriver release specifically for version 100 (Opera Stable 86)
    • operadriver v.99.0.4844.51 (Opera Stable 85)
    • operadriver v.98.0.4758.82 (Opera Stable 84)
    • operadriver v.97.0.4692.71 (Opera Stable 83)
    • operadriver v.96.0.4664.45 (Opera Stable 82)
  • Google Chrome version 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, & 108 (updated version 97 binaries)
    • chromedriver 108.0.5359.22
    • chromedriver 107.0.5304.62
    • chromedriver 106.0.5249.61
    • chromedriver 105.0.5195.52
    • chromedriver 104.0.5112.79
    • chromedriver 103.0.5060.134
    • chromedriver 102.0.5005.61
    • chromedriver 101.0.4951.41
    • chromedriver 100.0.4896.60
    • chromedriver 99.0.4844.51
    • chromedriver 98.0.4758.102
    • chromedriver 97.0.4692.71 (previously 97.0.4692.20)
  • Brave Browser version 96, 97, 98, 99, 102, 103, 104, 105, 106, & 107
    • bravedriver v.107.0.5304.88 (uses operadriver binaries)
    • bravedriver v.106.0.5249.119 (uses operadriver binaries)
    • bravedriver v.105.0.5195.102 (uses operadriver binaries)
    • bravedriver v.104.0.5112.81 (uses operadriver binaries)
    • bravedriver v.103.0.5060.66 (uses operadriver binaries)
    • bravedriver v.102.0.5005.61 (uses operadriver binaries)
    • there was no operadriver release specifically for version 101
    • there was no operadriver release specifically for version 100
    • bravedriver v.99.0.4844.51 (uses operadriver binaries)
    • bravedriver v.98.0.4758.82 (uses operadriver binaries)
    • bravedriver v.97.0.4692.71 (uses operadriver binaries)
    • bravedriver v.96.0.4664.45 (uses operadriver binaries)
  • Microsoft Edge version 100, 101, 102, 103, 104, 105, 106, 107, 108, & 109 (updated version 96, 97, & 98 binaries)
    • msedgedriver 109.0.1481.0
    • msedgedriver 108.0.1462.15
    • msedgedriver 107.0.1418.42
    • msedgedriver 106.0.1370.52
    • msedgedriver 105.0.1343.53
    • msedgedriver 104.0.1293.91
    • msedgedriver 103.0.1264.77
    • msedgedriver 102.0.1245.62
    • msedgedriver 101.0.1210.53
    • msedgedriver 100.0.1185.60
    • there was no msedgedriver release specifically for version 99
    • msedgedriver 98.0.1085.0 (previously 98.0.1086.0)
    • msedgedriver 97.0.1072.76 (previously 97.0.1072.8)
    • msedgedriver 96.0.1054.75 (previously 96.0.1054.26)
• MINOR BUGFIXES
  • Update URL for Quanta Magazine channel (commit 06fa9d8fd0ae52022912aa93cb313363e248ff6e)
  • Update time duration for video 130 in test reference files (commit b8641f75288192152e22f010fbf79281c523c4fb)
  • Use call command to properly run helper batch script (commit d519edfc83f2a06eb5ca507a7cd2f485ffc68b63)
  • Make browser version detection more robust
    • commit 4fbaa6f8794b3be89a45258fa6593ca10cc06d4f
    • commit 9b8ba0177271ac5243fc57898b6498fecc44abba
    • commit 90c532208ffbea1df33d8523de4d368ac205fbb0
    • commit c297e21dbb234ab13756e7fc4bc503448da21195
    • commit 02584be29e185b213a9f26ad7fdb0545a1f1ec65
• INTERNAL IMPROVEMENTS
  • Update save_thread_result package dependency version number → 0.0.9 (commit b5a9f14b51fd5219fdcc2eebfa38ce32fd20a640)
  • Support browser versions up to 120 (commit a155c05d3d6189efe0551590c155badb68b24994)

• BUGFIXES

  • commit 62464aa9cf803c8f8b45bbffcb748f691595bf1c
  • commit 93029fababed9506677505e9485197c9f4ce9498

• FEATURE IMPROVEMENTS

  • see commits with a commit message starting with "Raise"
  • also, see commit d43ef6a7b0721e2b8c1660fd2a2255ac9c8dbd71
  • see the following commits for more details:
    • commit e6057bba68bee85db57c66209a49fdc4e6d7c6fa
    • commit 784ef61b41269acde38b8041d937ab1386c244b7
    • commit 57c6a99bf7ce4e7a1881b20a07cbd59005f5b838
  • see the following commits for more details:
    • commit 67f519602e6d05913d5554138ad5a14ce639540b
    • commit cdcb5f48acb87e5fd7b1271d7e62f666013e1821
    • commit 01314c63f043af617bbbf07d6ac073b4a88fd0c3
    • commit c146baf39efa3866723a25573e02e29dd3e2296a
    • commit a5fddafca47e791a76b8e75147cff2e54c5e2385
    • commit 77ffc95237fb37676c2b4a2b2042c8df4f9107c9
    • commit 136dba035c43b3bb0427c04c4894406c2899ee8f
    • commit 912e54a6d1ba9a1e7594078d63d42d530daf91d0
    • commit c94a5a6594a03bbbdbb58d20a4a6bc7e9ee1aa23
  • see commits with a commit message
    • starting with "Include real time"
    • including log_time_taken

• PERFORMANCE IMPROVEMENTS

  • see commit 67d94a0886fbcd1a200694926dd40ca3a466cb14 for more details
    • NOTE: create_thread_from mentioned in this commit message was a typo and should be create_list_from
  • see the following commits for related changes:
    • commit 22a77d1bae6f3d9cc97ba463506b08fc9bdc7a3b
    • commit 41d6a962d7f7342fd885f93645d5f25eec0441cf
    • commit 67d94a0886fbcd1a200694926dd40ca3a466cb14
    • commit b3a902d2ff8b6bdd823449e9b8f8a3ee794dfefb
    • commit 4a07eff4b33c6d661442b5025f3ccff54d9807d7
    • commit 53cfd1a1b73578497e0d680194d5280117262d34
    • commit e46ebf84309d390d9b708b04559646a0d11dcc35
    • commit 5062cef05086e7d6cde3147364c4153e3e0ec073
    • commit b56720a2cfd9a28bc8ab77e8397e31861391607a
    • commit ff0806bfbe6f178854b1456160a0a9833b019b1e
    • commit 6aa54715ab85a1d79e5bdad95fd46b73568cfd31
    • commit d6c6e1a5c17b59f782eece2190c0a5e072001d51
    • commit cb4485be5c13aa18d2b475fda68a78ccf255ae47
    • commit 9f6049e4dbb7b49ead7da44e8f97986e94e437f5
    • commit 99380adb738c41fcf8767a4f43433e64a49f7666
    • commit c7f4ab86912f56672c025cf952141da491608d37
    • commit e182962a46aa94ae2018197b36343a86e1f8146a
    • commit ee1094a25e5741cb0ec29c78dba2b9a5984ce738
    • commit 5c5ef4ed6f79370ac180960371de6dfce1cd66d3

• INTERNAL IMPROVEMENTS

  • interesting changes
    • commit 96398e7f22fad73c9117acf3f22885eba2c73e35
    • commit 1211f1a4216e35fe2b3860ffc94469f8acd9f15b
    • commit 34cc678578acbc1bdfe090ad5bff936d913c0dd7
    • commit 7f4d28a0fae0a10cb6ea6334a5161312f54172c0
    • commit b84795ca237728f35dfd297623ddb3d9ff7bf5d8
    • commit 3bdd29a4b40e39e040f73bb28dba913f9e0b6fa3
  • displayed debugging information changes
    • commit cea29fee822e0cbbf4aa77a38b1d788dc3e59b5b
    • commit f838764d1a862c4d29ab57451f8d0b3bfb7bb075
    • commit 3da2ce66c9a200bd3c2064cf19d15c463c5fa6bf
  • testing/building changes
    • commit 155c06e1633276e6039316a528a3f514e4ed6d04
    • commit b5177051d6287bdca8e30bc88ef5c93dc21040e2
      • bug from commit 773fcb613d2f93946419143a051413a15dc6224c
    • commit 1f80d6b009965bc517bb8cab7d77f9558c8376bc
    • commit 3cf637f37555ab65e902734b17f5a5febc1f5ec2
    • commit 23cec086011c150c6c119fa3a1385a48c3e4ee3b
    • commit 512e7a7d024c02f5fc21160fbaab745f26990d5b
    • commit 90eecb3be4dd1658efe249bd10240cd99f0737ad
    • commit 1c14e1dc2bf68d0a2d546cb56058c4f4287bd8eb
    • commit 9d32a8fb2b810da97aded35dcb7492c2488f627c
    • commit 8e54376d43ef3b2e7b13f8b0eb22b676c609f3f9
    • commit 722eeb76e2bd406dac33b15a0026940babed8ccf
    • commit 02f92d87c979b99bb9fcbbaa11be8e03fb65c56e
    • commit e9c6fb3bf369cae6317fe61adc8daddd65e39a5b
  • refactoring changes
    • rename variables to be more descriptive
    • rename functions to be more descriptive
    • reorganize code for readability
    • remove unneccesary intermediate variables
    • add intermediate variables for clarity
  • documentation changes
    • improve error messages
    • improve README
    • improve docstrings

• BINARY UPDATES
• Mozilla Firefox
  • geckodriver v0.30.0 (Firefox versions ≥ 92)
• Opera Stable 77, 78, 79, 80, & 81
  • operadriver v.95.0.4638.54 (Opera Stable 81)
  • operadriver v.94.0.4606.61 (Opera Stable 80)
  • operadriver v.93.0.4577.63 (Opera Stable 79)
  • operadriver v.92.0.4515.107 (Opera Stable 78)
  • operadriver v.91.0.4472.77 (Opera Stable 77)
• Google Chrome version 92, 93, 94, 95, 6, & 97 (updated version 91 binaries)
  • chromedriver 97.0.4692.20
  • chromedriver 96.0.4664.45
  • chromedriver 95.0.4638.69
  • chromedriver 94.0.4606.113
  • chromedriver 93.0.4577.63
  • chromedriver 92.0.4515.107
  • chromedriver 91.0.4472.101 (previously 91.0.4472.19)
• Brave Browser version 91, 92, 93, 94, & 95
  • operadriver v.95.0.4638.54 (uses operadriver binaries)
  • operadriver v.94.0.4606.61 (uses operadriver binaries)
  • operadriver v.93.0.4577.63 (uses operadriver binaries)
  • operadriver v.92.0.4515.107 (uses operadriver binaries)
  • operadriver v.91.0.4472.77 (uses operadriver binaries)
• Microsoft Edge version 93, 94, 95, 96, 97, & 98 (updated version 90, 91, & 92 binaries)
  • msedgedriver 98.0.1086.0
  • msedgedriver 97.0.1072.8
  • msedgedriver 96.0.1054.26
  • msedgedriver 95.0.1020.53
  • msedgedriver 94.0.992.58
  • msedgedriver 93.0.961.52
  • msedgedriver 92.0.902.84 (previously 92.0.881.0)
  • msedgedriver 91.0.864.71 (previously 91.0.864.19)
  • msedgedriver 90.0.818.66 (previously 90.0.818.56)
• MINOR BUGFIXES
  • handle videos with no "Video Duration" field (commit 2f538e14e574f9435c13a544fb3018d347890f4a)
    • this is an extremely rare edge case
      • based on anecdotal data, occurs about 1 in every 70000 videos
  • update URLs shown in exception messages (commit 3f0961283836713cbc65592c6171693dcc808705 & commit 99ed682abec835304664e40e18d220fa0d4204b9)
  • correctly handle unfinished threads in create_list_from() method (commit aa4ff3de648f84891a96a5e7a33a8efb00fc0b19)
  • generalize URL normalization for removing trailing parameters (commit 0789a3e66b9965f4d29259c23860488714d81a3a)
    • this removes any trailing tracking parameters that might be associated with a video URL
      • e.g. youtube.com/watch?v=abcdefghijk?pp=sAQB → youtube.com/watch?v=abcdefghijk
  • verify page has videos (commit 82a48563a1a6bcdcac944d51516dcec079d0691e)
    • prevents crashing on channels with 0 public videos
• LOGGING IMPROVEMENTS
  • include total number of videos in each file after each run
    • commit 83827694160542cba28e1e7f7b24e6c5526ece1d
    • commit 3282089af25b2188206b0457475d55e21fcd0689
    • commit 94f5acb7e2b5ac2d0d41311aa1e9847665ff401e
    • commit c9f2671ae6d827c3b9d35013016610bda799590e
• INTERNAL CHANGES
  • refactor code to:
    • reduce code duplication
    • make variable and function names more context specific
    • place repeated code inside variables
    • make browser naming more specific (commit 81144cb88f5ac8c99a9fe27198b6cebf9bafdde2)

• see the following commits for more details:
  • commit e21a79ccc27c7cdc8741ade01162012e1cc718a8
  • commit 3e6262d3166c41350f300cbfca423197d689c627
  • commit 0c817f228041e91a56ff0345bc157752125f5edf
  • commit 5453b14719eb241e08f01d0c1fe5af96f1e4b6ef

• BREAKING CHANGE

  • BEFORE:
    • create_list_for() returned a str containing the name of the file the program wrote to
  • NOW:
    • create_list_for() returns a tuple containing
      • a list of lists containing the video information found by the program for the current run
        • by default, returns dummy video data to avoid cluttering the output
        • to return the actual video data, set the video_data_returned ListCreator attribute to True
          • dummy data: [[0, '', '', '']]
      • a tuple containing a str with the name of the channel (taken from the channel's heading) and a str with the name of the file written to
        • ('The Channel Name', 'the_name_of_the_file')
        • ('The Channel Name', '') if the ListCreator attributes are txt=False, csv=False, md=False, AND video_data_returned=True
    • see the NEW FEATURES section below for more details about video_data_returned
  • access the full documentation for the updated create_list_for method with help(ListCreator.create_list_for) in the python interpreter

• BUGFIX

  • fixes cookie_consent blocking logic for new HTML in GDPR regions
    • YouTube updated the HTML formatting for blocking cookie consent, and the previous cookie consent blocking logic broke
    • this release fixes the blocking logic to work with the new HTML formatting

• NEW FEATURES

  • overview for the new ListCreator attributes given here, but run help(ListCreator) in the python interpreter or read the "More API information" section in the python README to see the full documentation:
    • file_suffix allows more control over the file naming (True by default)
    • all_video_data_in_memory scrapes the ENTIRE YouTube channel's videos page, EVEN if files exist for the channel already (False by default)
      • must also set the video_data_returned attribute to True to actually get this information
    • video_data_returned returns the video data for all videos the program scraped (False by default)
      • data returned depends on a number of factors, see full documentation for more details
    • video_id_only saves only the video ID instead of the entire URL (False by default)
      • example: saves 'abcdefghijk' instead of 'https://www.youtube.com/watch?v=abcdefghijk'
  • overview for the updated file_name argument options in the create_list_for method given here, but run help(ListCreator.create_list_for) in the python interpreter to see the full documentation:
    • file_name='auto' names the output file(s) using the name that shows up under the banner when you navigate to the channel's homepage (with spaces removed)
    • file_name='id' names the output file(s) using the identifier from the URL provided to the url argument
      • run help(ListCreator.create_list_for) for a comprehensive list of examples
      • using file_name='id' is very useful when multiple channels have the SAME channel name

• PERFORMANCE IMPROVEMENTS

  • BEFORE:
    • the program pulled the video data from the selenium instance and wrote to the file(s) directly
  • NOW:
    • the program loads the video data from the selenium instance into memory, THEN writes the saved video data from memory to the file(s)
      • the performance improvement is more noticeable when writing more information
        • for example:
          • writing information for 200 videos to just a csv file: negligible performance difference between writing to csv file directly and loading to memory & THEN writing to csv file
          • writing information for 200 videos to csv, txt, md files: slight performance difference between writing to files directly and loading to memory & THEN writing to files, but still not much of a performance difference
          • writing information for 20000 videos to just a csv file: noticeable performance difference between writing to csv file directly and loading to memory & THEN writing to csv file
          • writing information for 20000 videos to csv, txt, md files: significant performance difference between writing to to files directly and loading to memory & THEN writing to files
        • summary:
          • the performance difference between writing to ONE file directly and loading to memory & THEN writing to ONE file is barely noticeable for small jobs and more noticeable for larger jobs
          • the performance difference between writing to MULTIPLE files directly and loading to memory & THEN writing to MULTIPLE file is more noticeable for small jobs (compared to writing to only ONE file) and SIGNIFICANT for larger jobs
  • logs from tests used to benchmark performance included below:

• to create the file:

It took 9.240757292005583            seconds to find 230 videos from https://www.youtube.com/user/schafer5/videos
It took 4.265756259999762            seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.csv
This program took 19.537945401003526 seconds to complete.

• to update the file:

It took 0.8453300589972059          seconds to find 60 videos from https://www.youtube.com/user/schafer5/videos
It took 0.6392399440010195          seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.csv
This program took 7.754261410002073 seconds to complete.

• to create the file:

It took 9.163404727999989            seconds to find 230 videos from https://www.youtube.com/user/schafer5/videos
It took 4.260267737000007            seconds to load information for 230 videos into memory
It took 0.002389371999996115         seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.csv
This program took 19.483281371000004 seconds to complete.

• to update the file:

It took 0.8521808300000089          seconds to find 60 videos from https://www.youtube.com/user/schafer5/videos
It took 1.0964175420000117          seconds to load information for 60 videos into memory
It took 0.0015745449999826633       seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.csv
This program took 7.985743492000012 seconds to complete.

• to create the files:

It took 9.166668037003546            seconds to find 230 videos from https://www.youtube.com/user/schafer5/videos
It took 10.160974278995127           seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.txt
It took 10.164936708999448           seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.csv
It took 10.168633003995637           seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.md
This program took 25.594990328005224 seconds to complete.

• to update the files:

It took 0.8503098270011833          seconds to find 60 videos from https://www.youtube.com/user/schafer5/videos
It took 1.5225159670007997          seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.csv
It took 1.5322243859991431          seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.txt
It took 1.5359413480036892          seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.md
This program took 8.472728426997492 seconds to complete.

• to create the files:

It took 9.367390958000005      seconds to find 230 videos from https://www.youtube.com/user/schafer5/videos
It took 4.218187391999997      seconds to load information for 230 videos into memory
It took 0.003894963000000473   seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.md
It took 0.005060710999998719   seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.csv
It took 0.006283445999997639   seconds to write all 230 videos to CoreySchafer_reverse_chronological_videos_list.txt
This program took 18.754924324 seconds to complete.

• to update the files:

It took 0.8672965029999986          seconds to find 60 videos from https://www.youtube.com/user/schafer5/videos
It took 1.0901944209999996          seconds to load information for 60 videos into memory
It took 0.005667658999996661        seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.csv
It took 0.008393589000000645        seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.txt
It took 0.008197031000001687        seconds to write the 0 ***NEW*** videos to the pre-existing CoreySchafer_reverse_chronological_videos_list.md
This program took 8.090583961999997 seconds to complete.

• to create the file:

It took 322.72226654399856          seconds to find 8095 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 256.63442500399833          seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.csv
This program took 585.4076739919983 seconds to complete.

• to update the file:

It took 0.8482559289986966          seconds to find 60 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 0.5600300389996846          seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.csv
This program took 7.653723870003887 seconds to complete.

• to create the file:

It took 316.9717323640002       seconds to find 8095 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 248.92245618300012      seconds to load information for 8095 videos into memory
It took 0.07691853599999376     seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.csv
This program took 572.114162118 seconds to complete.

• to update the file:

It took 0.8459371520000332          seconds to find 60 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 0.9670944140000302          seconds to load information for 60 videos into memory
It took 0.02941359300007207         seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.csv
This program took 8.209143252000104 seconds to complete.

• to create the files:

It took 314.01985485899786          seconds to find 8095 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 519.1903085960002           seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.txt
It took 519.1941804189992           seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.csv
It took 519.197644068001            seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.md
This program took 839.4073893879977 seconds to complete.

• to update the files:

It took 0.8488957250010571          seconds to find 60 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 1.580211615000735           seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.csv
It took 1.681963879003888           seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.txt
It took 1.6842712280049454          seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.md
This program took 8.823843261001457 seconds to complete.

• to create the files:

It took 316.342601403           seconds to find 8095 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 261.87072707100003      seconds to load information for 8095 videos into memory
It took 0.1363127509999913      seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.csv
It took 0.1775351439999895      seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.md
It took 0.18588107000005039     seconds to write all 8095 videos to KhanAcademy_reverse_chronological_videos_list.txt
This program took 584.703847726 seconds to complete.

• to update the files:

It took 0.8483775499998956          seconds to find 60 videos from https://www.youtube.com/c/KhanAcademy/videos
It took 1.0671216570001434          seconds to load information for 60 videos into memory
It took 0.17331316700006028         seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.csv
It took 0.22995445900005507         seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.txt
It took 0.23345572800008085         seconds to write the 0 ***NEW*** videos to the pre-existing KhanAcademy_reverse_chronological_videos_list.md
This program took 8.503321469999833 seconds to complete.

• to create the file:

It took 3420.0639533489993          seconds to find 32347 videos from https://www.youtube.com/user/NBCNews/videos
It took 4988.648231769999           seconds to write all 32347 videos to NBCNews_reverse_chronological_videos_list.csv
This program took 8414.909623333002 seconds to complete.

• to update the file:

# forgot to run this test :D

• to create the file:

It took 3367.386001154002       seconds to find 32357 videos from https://www.youtube.com/user/NBCNews/videos
It took 4880.191474030002       seconds to load information for 32357 videos into memory
It took 0.24478799300050014     seconds to write all 32357 videos to NBCNews_reverse_chronological_videos_list.csv
This program took 8253.73690525 seconds to complete.

• to update the file:

It took 0.8474488579995523          seconds to find 60 videos from https://www.youtube.com/user/NBCNews/videos
It took 1.1012943870009622          seconds to load information for 60 videos into memory
It took 0.11654774600174278         seconds to write the 5 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.csv
This program took 8.668505469999218 seconds to complete.

• to create the files:

It took 3396.025502143               seconds to find 32347 videos from https://www.youtube.com/user/NBCNews/videos
It took 7683.585577874001            seconds to write all 32347 videos to NBCNews_reverse_chronological_videos_list.txt
It took 7683.592947972               seconds to write all 32347 videos to NBCNews_reverse_chronological_videos_list.md
It took 7684.030176524999            seconds to write all 32347 videos to NBCNews_reverse_chronological_videos_list.csv
This program took 11086.336240618999 seconds to complete.

• to update the files:

It took 0.8738655359993572          seconds to find 60 videos from https://www.youtube.com/user/NBCNews/videos
It took 1.8775347520004289          seconds to write the 0 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.csv
It took 2.120259861001614           seconds to write the 0 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.txt
It took 2.132926509999379           seconds to write the 0 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.md
This program took 9.435579917999348 seconds to complete.

• to create the files:

It took 3478.1540728540003          seconds to find 32353 videos from https://www.youtube.com/user/NBCNews/videos
It took 5022.493407319              seconds to load information for 32353 videos into memory
It took 0.5065521739998076          seconds to write the 6 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.csv
It took 0.587243801997829           seconds to write all 32353 videos to NBCNews_reverse_chronological_videos_list.txt
It took 0.6058889249979984          seconds to write all 32353 videos to NBCNews_reverse_chronological_videos_list.md
This program took 8507.703900004002 seconds to complete.

• to update the files:

It took 0.8569685050024418         seconds to find 60 videos from https://www.youtube.com/user/NBCNews/videos
It took 1.1060196290018212         seconds to load information for 60 videos into memory
It took 0.5880495099991094         seconds to write the 4 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.csv
It took 0.8386826800015115         seconds to write the 4 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.txt
It took 0.8496009250011411         seconds to write the 4 ***NEW*** videos to the pre-existing NBCNews_reverse_chronological_videos_list.md
This program took 9.45503293100046 seconds to complete.

• compare changes to previous version
• if you are an existing user, skim through the BREAKING CHANGE and NON-BREAKING CHANGES sections below
  • if you are a new user, you do not need to worry about these sections - just skip to the NEW FEATURES section at the bottom and read the python README to get started
• BREAKING CHANGE
  • the program now extracts the video duration for every video uploaded by a channel
    • this will likely cause problems when updating pre-existing csv files, since
      • the video duration information goes in a new column
      • csv file renderers expect consistent column formatting throughout the file
        • BUT a pre-existing csv file will only have the Video Number,Video Title,Video URL,Watched,Watch again later,Notes columns
        • so updating a pre-existing csv file will result in newly extracted videos having the Video Number,Video Title,Video Duration,Video URL,Watched,Watch again later,Notes columns while the already extracted videos will only have the Video Number,Video Title,Video URL,Watched,Watch again later,Notes columns (no Video Duration column)
        • therefore, updating a pre-existing csv file will result in the newly extracted videos having 7 columns, while pre-existing videos will have only 6 columns
    • if you want to continue using your pre-existing csv file and do NOT WANT TO INCLUDE the video duration for previously extracted videos:
      • if you have NOT yet updated the pre-existing csv file:
        • APPROACH 1: use a csv file editor such as Excel, Google Sheets, Numbers, IDE extension, etc.
          • open the csv file
          • insert the Video Duration column between the Video Title and Video URL columns
          • save the file
            • the csv editor should automatically format the existing rows to include the Video Duration column
            • therefore, all rows should now have an empty cell for the Video Duration column
        • APPROACH 2: use a simple text editor/IDE
          • open the csv file
          • insert the Video Duration column between the Video Title and Video URL columns
          • text editors will NOT automatically format the existing rows to include the Video Duration column
            • so you will need to manually format the existing rows to include the Video Duration column
            • the simplest way to do this would be to use a Find and Replace operation:
              • Find all occurrences of: ,https://
              • Replace with: ,,https://
                • this assumes the only urls in the csv file are in the Video URL column!
                  • if you have manually added/modified parts of the file and this is no longer true, you will have to modify this approach slightly to meet your needs
      • if you have ALREADY updated the pre-existing csv file:
        • you will not be able to use APPROACH 1 from above
        • you will need to use APPROACH 2 with slight modifications:
          • Find all occurrences of (with regular expression mode enabled): ([^:][^\d]{2}),https://
          • Replace with: $1,,https:// (depending on your editor, you may need to substitute $1 with \1 or something else)
            • looks for ,https:// where it is NOT preceeded with :\d\d
              • since the most recently extracted videos will have the video duration but the already existing videos will not have the video duration
              • so this only adds a comma for previously extracted videos without the video duration
              • as with APPROACH 1, this also assumes the only urls in the csv file are in the Video URL column!
                • if you have manually added/modified parts of the file and this is no longer true, you will also have to modify this approach slightly to meet your needs
          • if the file is a chronological_videos_list file (as opposed to a reverse_chronological_videos_list file):
            • you will ALSO need to insert the Video Duration column between the Video Title and Video URL columns in the csv header
              • since chronological_videos_list files use the csv header from the pre-existing csv file
                • NOTE the program updates the reverse_chronological_videos_list csv header every time the program looks for new videos when rerun on a previously scraped channel
                • but usually this csv header update is not noticeable since the header does not change
                • the csv header update is noticeable this time, however, since there is a new column (Video Duration)
                • for chronological_videos_list files, however, the program never updates the csv header
    • if you want to continue using your pre-existing csv file and WANT TO INCLUDE the the video duration for previously extracted videos:
      • rerun the program for the channel (in a different directory)
      • copy over any notes you took in the pre-existing file to the new file with the video duration information
    • if you do NOT want/care about using the pre-existing csv file
      • just delete the pre-existing csv file and rerun the program on the channel again (or run the program on the same channel from a different directory)
        • NOTE that if the channel deleted a video OR unlisted a video between
          • the time the video information was originally scraped
          • and you rerunning this after installing release 0.6.0+
          • the deleted/unlisted video(s) will not show up (no workaround for this - this is how YouTube displays videos)
• NON-BREAKING CHANGES
  • txt and md files now also include the video duration information
    • this is simply an extra line in the output file, and will not cause any rendering issues since txt and md files do not depend on a consistent formatting the way csv files do
  • txt and md file now use slightly different formatting such as
    • fewer newlines
    • md files using h3 headings for video information instead of bullet points (the bullet points were also improperly formatted previously, but since they are no longer used, this is not an issue)
  • NOTE that if you want these files to contain the video duration information, you will still need to rerun the program on the channel from scratch (either in a different directory, or after deleting the pre-existing files in the current directory)
• NEW FEATURES
  • verify_page_bottom_n_times attribute
    • for more information, see
      • commit a68f8f62e5c343cbb0641125e271bb96cc4f0750
      • commit 5b361de36f4d38704d4f4f5cb079162ace2bcb6c
      • commit 916f0502420f9f9ccc64b370cf6f531cad6f24c7
      • commit 6a02bfeb177089d8c21c7ffe2d2c67a979dbc4b2 (documentation)
  • file_buffering attribute
    • for more information, see
      • commit 0730cdb6852c064bb667ab3df22f52830ad8e065
      • commit 38b83177f867e1cf796e78be0efaabe2359b800c (documentation)

• compare changes to previous version
• creates new file I/O threads if writing to more than 1 file
  • see commit 58c5faba14da25b89e104a50d380489a30d8df71 for details
• supports scraping multiple channels from a txt file containing urls
  • see Scraping multiple channels from a file simultaneously with multi-threading section in python README for usage details
  • see __init__.py file for code changes

• compare changes to previous version
• no changes in API or functionality
  • these changes should have been a part of release 0.5.7, but missed these bugs during testing
• see the following commits for bug fixes:
  • commit b41081485c3d599856f4431bcee01e6bb79146da
  • commit aa3b6243883bc22a1c52ef09639de0c8b50d40b9
  • commit cd65c5c73d945db487743b4679f2997a6f1d06e4
  • commit e5d16d7a87b6907789615776b12eb68d860524e4

• compare changes to previous version
• this release automatically blocks cookies (default) or accepts cookies if YouTube redirects you to consent.youtube.com
  • this is the Before you continue to YouTube Cookie consent page discussed in the discussion Problem with cookies. (#14) as well as in the issue Before you continue to YouTube - Cookie consent (#16)
  • this is possible through the new cookie_consent parameter for ListCreator:

lc = ListCreator()                    # cookie_consent=False by default
# OR
lc = ListCreator(cookie_consent=True) # if you want to accept cookies, or if blocking cookies doesn't work properly

# rest of API unchanged
# use other code as you normally would
# .....

• compare changes to previous version
• Mozilla Firefox
  • geckodriver v0.29.1 (Firefox versions ≥ 87)
• Opera Stable 75, 76 & older binaries
  • operadriver v.90.0.4430.85 (Opera Stable 76)
  • operadriver v.89.0.4389.82 (Opera Stable 75)
  • operadriver v.2.36 (Opera 53)
  • operadriver v.2.35 (Opera 52)
  • operadriver v.2.33 (Opera 50)
  • operadriver v.2.32 (Opera 49)
  • operadriver v.2.30 (Opera 48)
  • operadriver v.2.29 (Opera 46)
• Google Chrome version 89, 90, 91 & older binaries
  • chromedriver 91.0.4472.19
  • chromedriver 90.0.4430.24
  • chromedriver 89.0.4389.23
  • chromedriver 2.45 (v70-72)
  • chromedriver 2.44 (v69-71)
  • chromedriver 2.43 (v69-71) - use chromedriver 2.44
  • chromedriver 2.42 (v68-70)
  • chromedriver 2.41 (v67-69)
• Brave Browser version 89, 90
  • bravedriver v.90.0.4430.85 (uses operadriver binaries)
  • bravedriver v.89.0.4389.82 (uses operadriver binaries)
• Microsoft Edge version 91, 92 (updated version 89 & 90 binaries)
  • msedgedriver 92.0.881.0
  • msedgedriver 91.0.864.19
  • msedgedriver 90.0.818.56 (previously 90.0.803.0)
  • msedgedriver 89.0.774.77 (previously 89.0.774.27)

• compare changes to previous version
• no updates to API and no new features or functionality
• makes package structure simpler following code deduplication (moves shared code nested under the file subpackage in create_file.py and update_file.py into scroller.py and write.py to ↑ DRY and make things Easier to Change (ETC))
• significantly simplifies logic to create webdriver instance (commit d90c29f7b8d117a9a1f600219d286ca3240c8207)
• renames function names and module names (commit 490bd764a1e205582005ec62f58955c3ebfe0697 and commit cafca0af270fbd11e77d841f2426dbcd3f49b524) to be consistent with naming guidelines highlighted by Robert C. Martin in chapter 2 of Clean Code: A Handbook of Agile Software Craftmanship
  • use a noun/noun phrase for a class (applied to module names in this codebase as well) and a verb or verb phrase for a function/method
• reduces size of yt-videos-list (shipped) package with minifier.py updates

• commit ad5cf3c247389fff1b110c976ddeeb91fddde212
• commit b610178d1974093cb7ba0106e02ec365f9a735e0
• commit 8fedafe9a510efc117ab283648fb21f404f3a612
• commit eb840c20d1faca72228097877fd819b57b772833
• commit 615166fff90667119d55b3917af2d84c746b7ea9
• commit 8326953bfa668f9775efdd628cb1bd894d38c99d
• commit 263ae023c65460303152af035b670aabc819f9c7
• commit 7f476f6ed3910d81dc7ea36bb10140a58a219491
• commit b13120e2b884c7382953b2dc581f2ac1a966b43d
• commit f024701f9ec633d682e4033c07851fad609a533c
• commit 6ae764086f6ce4fdf4bce255f0dc5354de8a36aa
• commit 490bd764a1e205582005ec62f58955c3ebfe0697

• compare changes to previous version
• Firefox
  • geckodriver v0.29.0 (Firefox versions ≥ 84)
• Opera Stable 74
  • operadriver v.88.0.4324.104
• Google Chrome version 88
  • chromedriver 88.0.4324.96
• Brave Browser version 88
  • bravedriver v.88.0.4324.104 (uses operadriver binaries)
• Microsoft Edge version 90 (updated version 88 & 89 binaries)
  • msedgedriver 90.0.803.0
  • msedgedriver 89.0.774.27 (previously 89.0.759.0)
  • msedgedriver 88.0.705.74 (previously 88.0.705.29)

• compare changes to previous version
• prior to this release, the program scrolled all the way to the bottom of the channel's video page if txt, csv, AND md were not set to True
• with this release, the program only scrolls as far as needed until it hits videos that are already in the file(s) to be updated

• compare changes to previous version
• Firefox
  • geckodriver v0.27.0 (Firefox versions ≥ 82)
  • geckodriver v0.28.0 (Firefox versions ≥ 78)
• Opera Stable 73
  • operadriver v.87.0.4280.67
• Google Cgrome version 87
  • chromedriver 87.0.4280.88
• Brave Browser version 87
  • bravedriver v.87.0.4280.67 (uses operadriver binaries)
• Microsoft Edge version 89 (updated versiom 88 binaries)
  • msedgedriver 89.0.759.0
  • msedgedriver 88.0.705.29 (previously 88.0.685.0

• compare changes to previous version

• compare changes to previous version
• major change includes changing output file name to include _reverse_chronological or _chronological
  • e.g.
    • MyChannel_reverse_chronological_videos_list.txt
    • MyChannel_chronological_videos_list.txt
      • compare with previous naming convention of
    • MyChannel_videos_list.txt - REGARDLESS of whether the file was in reverse chronological order or chronological order

• 'gbk' codec can't decode byte 0x80
  • https://github.com/slow-but-steady/yt_videos_list/issues/3
  • PR https://github.com/slow-but-steady/yt_videos_list/pull/5
    • commit 1fcf32a5f433ab8a82b9c56e90be6894eecf12e1: Explicitly use utf-8 for encoding and decoding all files
    • commit 4c4cc5a0c6d7f8ee0422e44cdfecd4d152d4adf5: Specify name,operation,encoding params for all file IO
• Can lc.create_list_for() return the csv file name?
  • https://github.com/slow-but-steady/yt_videos_list/issues/4
  • PR https://github.com/slow-but-steady/yt_videos_list/pull/6
    • commit a30be66e9ded08faa2f7bd13247cf1105078d62a: Return file name after lc.create_list_for() finishes
• Temporal filename "yt_videos_list_temp.txt"
  • https://github.com/slow-but-steady/yt_videos_list/issues/7
  • https://github.com/slow-but-steady/yt_videos_list/pull/8
    • commit 5d7bfada4bb14e407e01b3d8dfcef2fba7b64417: Name temp files using channel (for file creation)
    • commit c781a2f70455d3f7f7e28bd66f9234173b100aa2: Name temp files using channel (for file updates)
    • commit 3772d57465ef72567cf6d85f0e83fee72ca50f6e: Indicate type of temp file being written
  • append time to end of temp file name
    • initially appended UNIX time
      • commit d0f76e2e4205766d441209771a2cab1ee7c5231b: Append UNIX timestamp to temporary file name
      • commit 110912d39c8044621d168bcfa1c5c2c83b7164af: Replace the dot in timestamp with a dash (should have been included in commit above)
    • then changed appended time from UNIX time to ISO 8601 datetime format to increase readability
      • commit a349c5b4d7eb0325a655aeae598da6cb0834dfcc: Append ISO 8601 datetime instead of UNIX time to temp file name

• commit 8cf2e1555e0b7312d7a9e3eb8966c205482852c2: Append (reverse_)?chronological to file name
• commit 3285d930684afbfba82b9eec30ff4ce9448dfcc3: Update location for testing file paths
• commit dc38e2da3b27f41ae30320821a7ecfac23051cfa: Modify output file naming
• commit 90212e88e2e6285cd7efca44e7c1123117cf10d8: Simplify file suffix creation

• vertically align similar messages to facilitate quick comparisons between related messages
  • commit c87792755004fd0898a74bd7843172180f4cfeee: Vertically align logging output
  • commit e7458d036eb5527afdf4eec50223c93ba7ada55a: Make logging messages more visible
  • commit 34878884012ed5c0e220369c8a5946ea6e2d83e5: Right pad all testing log messages with ">"
  • commit 60cbee3916e226a6881d4ac1ba2d33e3a15669b1: Log thread being created (during testing)
  • commit 18146c49ca0360165906773711bea96d95a1a240: Rejustify program logging messages
  • commit 296fc4d7fc2a143f26e8220cfedf732ab5233b9c: log program info with custom logger helper module (↑ DRY)
  • commit ab188409af1480690596a341ff3675fef1c28349: Log "PROGRAM COMPLETE" instead of "PROGRAM COMPLETED"
• log datetime for every event
  • commit b00b0885439c2570057997a4288cb5c962704cd8: Print datetime during testing
  • commit e6101b89a9b1cd58281b05068a6369acb4d8f000: Log datetime while running program
• LOG ALL information to corresponding LOG FILE for channel
  • log file naming
    • commit 257e9e9fab06e541d2ec7fed9c165488d6c24bed: Name log file using ISO 8601 datetime
    • commit f53e6af8624b2c4d67f9a36ad67e1d47a529cbab: Name log file using output file name
• general logging
  • commit 2b6bb4f952833f15cd8bb2991e3e6bca64787582: Enable optional logging to user specified log_file
  • commit b1e784a5292f1deec5df07ea33c68a9e15bcdc1e: Log test output to "{suffix}.log" (testing)
  • commit 2b63e9c42f72e2727149f7b09a1075430d6ab36f: Enable INFO level logging by default
  • commit 82a0129d82ea67475af902cf4a8a07c016d853b4: Simplify logging via custom context manager text writer (EXTREMELY detailed!)
    • commit 3b6e3fcec63e7d76e9facfaf961d9b6a67dec52b: Pass logging_output_location to txt_writer()
  • commit 6513697c6a5dbf212ca16e1f7e2ef42d552738fe: Log program start & end messages instead of printing to console
  • commit add1f35fad70463dfc56547e686afb21e73c5df6: Log name of driver during testing
  • commit 21e6bde45e7966d5cebdb3ab94f35d7462007d9e: Add testing info to log files during tests
  • commit 9a41424151c89836928e75e2e0407c844e447781: Enable logging to multiple files during testing
  • commit 8bb10084efe6ce320ad07beb9faf1652287cf9b9: Simplify testing info logging
  • commit 9a41424151c89836928e75e2e0407c844e447781: Enable logging to multiple files during testing
  • commit 99d7be0896aa422bfe8138b7a882bd446e9ecc4b: Add "*" 200x when test starts to clearly divide log file
  • commit ca7f4c91f1c8294e85ad8f34e5222d057036de0f: Log thread name when new thread created (during testing)
  • commit 97ccc33716da2dc281c84989092f76b8b9741c3d: Log ">>>STARTING PROGRAM<<<"
  • commit 2e20d8a7f53a0a6af0d0b84d54e316558c055b48: Log ">>>PROGRAM COMPLETED<<<"
  • commit 8a3e4f290f35fb04d91e1b93da0c10571d3c4694: Log write & file renaming successes separately
  • commit dab5ecf27e26a5eec6878c83e90c67ee5487acf8: Move create_file.py & update_file.py decorator code → log_extraction_information()
  • commit fb8311869e89179dcf2bbf2849edcd5f97b216a1: Always log to log file but allow console logging muting
  • commit 56bc309e7229d3ad4eee6da3e42c1e7171c7e7b9: Log "video" if 1 new video found, otherwise log "videos"

• commit 82a0129d82ea67475af902cf4a8a07c016d853b4: Simplify logging via custom context manager text writer (also mentioned above, EXTREMELY detailed!)

• only occurs when
  • scraping the same channel on 2 threads with reverse_chronological set to True on one thread and False on the other thread
  • and starting both threads WITHIN a few tenths of a second of each other
  • WITH pre-existing files for both reverse_chronological file and chronological files but DIFFERENT number of videos in the files for reverse_chronological and the chronological files
• commit 97d928f8a9b73a7d7d4c6f02e49ff4f44fa37db6: Test pre-existing csv, txt, md files first
• commit 7bf88c10e7ebbb4f02be9fc2301026bdb4deb1c7: Modify partial chronological files (catch bug more frequently)
• commit e787c3a76352602b50d7db2a00b50a1962830107: Return visited videos sets instead of creating global variables

• commit 3b78b0a28cadff93cc7dc602c4c9b1df360acb67: Delete only relevant files before testing
• commit e2e1ae9f20c42de0feefffc80fc1961a8a00dd8e: Avoid starting new thread after last test case
• commit 7be64c3e177547734a1a918200362367b645e139: Explicitly check which thread ends first
• commit 5aafa78dcdef410d0ae3347876b980277cca6da2: Simplify threading logic for tests
• commit 930b59c19ed80f0ee3613e661c926abc633fe18b: Avoid multi-threading for safaridriver
• commit 4da6186f4a856b4097dc866c2dd81029e7333a84: Remove debugging print statements ("previous commit" refers to the commit above)
• commit fcd744da234a93937fddd3204b3a72c15e1f1177: Verify variable exists before printing message
• commit 9c2529d2981e2dab6685f8712cbebad6b4802dbf: Ensure threads finish before proceeding
• commit 27cc6a9fde087715c7179d6745b139daf3bb731e: Make thread checks more robust

• commit 6bbac4944c93ec22605cd23bad003a4135b50b83: Remove deprecated create_list_for() arguments

• commit 8fc62703047b9f8de287306239885cd5138a8d7e: Add custom class to store thread result
• commit f1d58f6deeb2becf9038a94c3fb964bccc5321d3: Make ThreadWithResult attribute names more descriptive
• commit b10480b6979f96443ab9e2e62e515c4da30eccdb: Add ThreadWithResult class docstring (test_shared.py)

• commit fd8ad485685768b3bdf918859e7ea514c1db2963: Point future drivers to newest available driver ("next commit" refers to commit below)
• commit fd878f3d339732f776da841ed53d22d77ec96f73: Indicate failed update may be due to new driver version

• commit e0569f2aced090f7555bedbfaf254d39454738f6: Create json file for download commands
  • previously the project only provided pseudo json in the yt_videos_list/docs/dependencies_pseudo_json.txt file

• started with
  • commit 6fa0deb3d8b6c73612d79ea927f81f62e1141c29: Run "pip" on Windows and "pip3" on Unix
    • following commit 8c73de6295edf461b64b594fef0f8a01470056bb: Make PATH_SLASH a global variable
• addressed with
  • commit 9550ca5b5c330238e94be8d4874fe0f4953731cb: Update local package without yt_videos_list submodule function
  • commit 878fb67585debc00200ce831ae2106ddd4c7acea: Remove duplicate import (test_cross_platform_drivers.py) (since function now imported from tests.determine module)
  • commit 0204dd2201cbb3b68ca3cac333312216a7d65041: Run pip install directly from test script
  • commit 829a1ae681215ca9b06e74dad435372aef108c91: Update local package if python test module called directly

# without yt_videos_list submodule function
for i in {1..10}; do (time (for i in {1..100}; do python3 minifier.py; done)); done

real	0m8.261s
user	0m5.433s
sys	0m2.259s

real	0m8.288s
user	0m5.429s
sys	0m2.247s

real	0m8.022s
user	0m5.272s
sys	0m2.164s

real	0m7.989s
user	0m5.266s
sys	0m2.165s

real	0m7.984s
user	0m5.253s
sys	0m2.163s

real	0m8.009s
user	0m5.268s
sys	0m2.164s

real	0m8.047s
user	0m5.269s
sys	0m2.175s

real	0m8.068s
user	0m5.242s
sys	0m2.182s

real	0m8.030s
user	0m5.289s
sys	0m2.164s

real	0m8.046s
user	0m5.284s
sys	0m2.176s

# with yt_videos_list submodule function
for i in {1..10}; do (time (for i in {1..100}; do python3 minifier.py; done)); done

real	1m28.987s
user	0m42.470s
sys	0m41.508s

real	1m28.921s
user	0m42.508s
sys	0m41.411s

real	1m28.753s
user	0m42.436s
sys	0m41.378s

real	1m29.467s
user	0m42.700s
sys	0m41.732s

real	1m28.672s
user	0m42.286s
sys	0m41.406s

real	1m28.415s
user	0m42.297s
sys	0m41.202s

real	1m28.629s
user	0m42.360s
sys	0m41.244s

real	1m29.088s
user	0m42.587s
sys	0m41.527s

real	1m29.392s
user	0m42.644s
sys	0m41.637s

real	1m29.345s
user	0m42.657s
sys	0m41.643s

# without yt_videos_list submodule function again
for i in {1..10}; do (time (for i in {1..100}; do python3 minifier.py; done)); done

real	0m8.488s
user	0m5.585s
sys	0m2.308s

real	0m8.293s
user	0m5.497s
sys	0m2.251s

real	0m8.115s
user	0m5.396s
sys	0m2.188s

real	0m8.116s
user	0m5.396s
sys	0m2.179s

real	0m8.145s
user	0m5.395s
sys	0m2.198s

real	0m8.066s
user	0m5.367s
sys	0m2.170s

real	0m8.042s
user	0m5.340s
sys	0m2.162s

real	0m8.029s
user	0m5.329s
sys	0m2.159s

real	0m8.170s
user	0m5.420s
sys	0m2.195s

real	0m8.154s
user	0m5.426s
sys	0m2.190s

• commit 1cdd8f5d203c895c01be4e7a8400afefd7c18c30: Revert "Make command consistent with other unix commands"
  • commit 6783c40e0dd97d16bd569370145be5f8c992bfd8: Make command consistent with other unix commands
    • following commit 76c066f2751acf9376e39575bd74bd779894b934: Move repeated commands into helper functions
• addressed in
  • commit d762b00acee4557f0de92bd17d5ea28f7b4ecb60: Remove rm /usr/local/bin/sha512_sum command (bravedriver)
  • commit e32f69ff1bf22f7c7da49d7b2bad55a06269eca8: Remove sha512 removal command for Windows bravedriver too

• commit 30e9701ed984122d61a28abc5b2598adcea60e93: Make global varaibles local

• compare changes to previous version
• Opera Stable 72
  • operadriver 86.0.4240.80
• Google Chrome version 86
  • chromedriver 86.0.4240.22
• Brave Browser version 86
  • bravedriver 86.0.4240.80 (uses operadriver binaries)
• Microsoft Edge version 88 (updated version 86 & 87 binaries)
  • msedgedriver 88.0.685.0
  • msedgedriver 87.0.669.0 (previously 87.0.661.0)
  • msedgedriver 86.0.622.58 (previously 86.0.622.31)

• compare changes to previous version
• Opera Stable 71
  • operadriver 85.0.4183.102
• Chrome version 85
  • chromedriver 85.0.4183.87
• Brave Browser version 85
  • Brave Browser version 85.0.4183.102 (uses operadriver binaries)
• Microsoft Edge version 87 (updated version 85 & 86 binaries)
  • msedgedriver 87.0.661.0
  • msedgedriver 86.0.622.31 (previously 86.0.606.0)
  • msedgedriver 86.0.622.31 (previously 85.0.564.30)