Project

General

Profile

Bug #325

TMDB Scraping issues - it only scrapes some titles

Added by tim619 over 4 years ago. Updated about 3 years ago.

Status:
New
Priority:
High
Assignee:
Category:
Scraper (Music/Video Metadata
Target version:
Start date:
09/21/2014
Due date:
% Done:

0%

Resolution:
Affected Version:

Description

The problem is that TMDB only scrapes some titles (and always the same!).
All others it ignores although it could scrape 98% of them (tested with manual scrape, see video.)

here is video demonstrating the process: https://mega.co.nz/#!wFMl1R5I!EziRDiG6kNyd-Rgx3aQkQdOq3dlTPKQuT__HDJtgw5w


Files

xbmc.log (184 KB) xbmc.log tim619, 09/21/2014 10:30 AM

History

#1

Updated by tim619 over 4 years ago

Good news to this: it is not an error of the scraper! I understood debug log and here is what it does:

If folder not in database:
if Found matching URL NFO file:
take url and fetch info
if result good:
continue with scraping of additional stuff like images
else:
die and go to next folder # Here we have our bad end

As an improvement I would suggest:
If folder not in database:
if Found matching URL NFO file:
take url and fetch info
if result good:
continue with scraping of additional stuff like images
else:
continue with name search the same way as there wouldnt be any nfo file. #This would fix #325

In other words: If I would delete all movie.nfo files it would success with good scrape rate.

If you can tell me were this algo is defined I can maybe fix this.

#2

Updated by tim619 over 4 years ago

Okay the file is obviously named "VideoInfoScanner.cpp"

#3

Updated by tim619 over 4 years ago

I'm sorry for the double posts, but I'm not able to edit.

The error is thrown because CNfoFile doesn't declare it as ERROR_NFO if url is bad.
In line 1517 you can find: else if (result != CNfoFile::NO_NFO && result != CNfoFile::ERROR_NFO)
it goes inside this although it has some kind of bad url or no url from nfo file.
Maybe NfoFile.cpp line 177 doesnt get all cases.

Hopefully you know some more about this code, but atm I'm glad that I could fix it with deleting nfo files ;)

#4

Updated by tim619 over 4 years ago

Today I deleted all nfo files from movie folders and it worked flawless as expected, so the problem is obvious.

#5

Updated by buzz over 4 years ago

  • Target version changed from 3.5 to Future / Pending
#6

Updated by tim619 about 3 years ago

I now made a deeper bug tracking and the problem in all cases is that if for a MOVIE_NAME.EXT a MOVIE_NAME.nfo exists AND
MOVIE_NAME.nfo has an imdb url included THAN in VideoInfoScanner.cpp the code


case CNfoFile::URL_NFO:
type = "URL";
break;

becomes valid. The problem is here that the scrapers cannot handle this imdb url but try to and not fall back to title search.
As a solution I would recommend to completely delete the type URL_NFO for a CNfoFile as it can only generate problems as far as I see.

Also available in: Atom PDF