Author Topic: Scraper issue  (Read 3344 times)

mgalaxy

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
    • Email
Re: Scraper issue
« Reply #25 on: February 07, 2019, 04:18:04 AM »
I can confirm that scraping of the whole romset finished without any issues.

Thanks again for the effort!
Thanks for reporting, we are always committed to the satisfaction of our users!

vonblubba

  • Newbie
  • *
  • Posts: 12
    • Email
Re: Scraper issue
« Reply #24 on: February 07, 2019, 03:45:24 AM »
I can confirm that scraping of the whole romset finished without any issues.

Thanks again for the effort!

vonblubba

  • Newbie
  • *
  • Posts: 12
    • Email
Re: Scraper issue
« Reply #23 on: February 02, 2019, 11:17:43 AM »
I started a new scraping process for the entire romset. It's been running for a couple of ours (a few hundreds roms scraped) without an issue. So far so good! I'll keep you posted on the final result.

Thanks for the effort, however this goes!

mgalaxy

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
    • Email
Re: Scraper issue
« Reply #22 on: February 01, 2019, 04:03:01 AM »
Please find this beta version that should fix all inconsistent stops while scraping from Screenscraper.
Simply extract over the mGalaxy folder and replace all existing files.
Please report here how successful or not it then gets! ;)
« Last Edit: February 02, 2019, 12:58:48 PM by mgalaxy »

SegaDoesWhatNintenDont

  • Newbie
  • *
  • Posts: 26
Re: Scraper issue
« Reply #21 on: January 30, 2019, 07:07:25 AM »
Quick update: I checked my roms on a different machine (running Win 7). I got only minor issues and one crash while I was able to scrape 700+ roms. So apparently mGalaxy doesn't play nice with my Win 10 setup for whatever reason...

SegaDoesWhatNintenDont

  • Newbie
  • *
  • Posts: 26
Re: Scraper issue
« Reply #20 on: January 29, 2019, 03:33:27 PM »
Hi, thanks for looking into it!

Here's one of the files that contain a comment. Unfortunately that alone does not seem to be the issue. I created a separate folder with  30 roms only and made sure to delete those with comments.. The crashes still occur.

As mentioned before it is not always the same rom. Usually the next time a rom will be scraped succesfully and the rom after that crashes the software. It's almost as if scraping all components at once is too much so that Runway needs several attempts per rom?
But it makes no difference if I select all components to be scraped or just the screenshots, for example.

Oh, the rom is 1,75 mb. Is there a way to send it? (Attachment  limit seems to be 1200 kb)

mgalaxy

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
    • Email
Re: Scraper issue
« Reply #19 on: January 29, 2019, 09:54:30 AM »
Still trying to isolate the 'bug' while scraping, all of a sudden, without modifying anything, everything is working properly (tested during the last 6 hours)...the problem might come from a change in the way the screenscraper database is serving files...I continue to check

mgalaxy

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
    • Email
Re: Scraper issue
« Reply #18 on: January 29, 2019, 09:51:13 AM »
One thing that makes Runway crash are comments in the zip files, apparently. This still doesn't change anything about the above mentioned behavior, but in case one zip contained the additional comment property Runway would crash on the same zip every time... since I'm not getting any comments.. should I open a new thread for this?
could you please send me such a zip file?

SegaDoesWhatNintenDont

  • Newbie
  • *
  • Posts: 26
Re: Scraper issue
« Reply #17 on: January 29, 2019, 09:44:30 AM »
Yes, the window just closes. No message.

I tested mGalaxy on a different PC now, using a fake rom folder with about 10 zipped text files. All of them showed up in the games list and had screenshot, videos and logos scraped... I know now that all the features CAN work and I'm capable of setting everything up, but it doesn't solve my problem of course..

My PC at home is quite fast, while the wifi is OK-ish. I will do a test tonight, unzipping all the roms and rezipping them, thus getting rid of any comment properties or whatever.

If that doesn't work I'll test the fake rom approach to see if my install of windows 10 is giving mGalaxy trouble.
One thing is for sure, though: I tested the same roms with Launchbox, which worked like a charm without any crashes every time. So even with possible workarounds it is still mGalaxy that would benefit from being more compatible or forgiving with whatever is the issue here...

vonblubba

  • Newbie
  • *
  • Posts: 12
    • Email
Re: Scraper issue
« Reply #16 on: January 29, 2019, 01:14:15 AM »
@SegaDoesWhatNintenDont so in your case it just crashes? No error messages whatsoever? It's probably a different issue then, I suppose. Does it manage to download any media file at all?

Good catch about the zip comments, though. I didn't think about that.

SegaDoesWhatNintenDont

  • Newbie
  • *
  • Posts: 26
Re: Scraper issue
« Reply #15 on: January 28, 2019, 04:32:53 PM »
One thing that makes Runway crash are comments in the zip files, apparently. This still doesn't change anything about the above mentioned behavior, but in case one zip contained the additional comment property Runway would crash on the same zip every time... since I'm not getting any comments.. should I open a new thread for this?

SegaDoesWhatNintenDont

  • Newbie
  • *
  • Posts: 26
Re: Scraper issue
« Reply #14 on: January 28, 2019, 01:28:39 PM »
I would like to contribute to this thread because I'm having constant crashes with mGalaxy_Runway 7.0 on Windows 10 Home. Here's what I did:

1. fresh install of mGalaxy 7.0
2. Setup paths to respective folders in Runway
3. clicked update using the GamesDB server (notification: submitted games 781)
4. clicked update with Screenscraper selected...

Runway quickly goes through all the rom names, then says updating info for file: "whatever game starting with A..", goes through two or three names, lingers on the last one for 10 seconds or so and then just closes without any alert.

5. Trying again. This time Runway stops at the first game and closes.
6. Trying again. This time Runway stops at the second game and closes....

Now I can play the game of restarting Runway and hitting update to have one more game updated with scraped info each time I launch the app...

vonblubba

  • Newbie
  • *
  • Posts: 12
    • Email
Re: Scraper issue
« Reply #13 on: January 27, 2019, 12:04:00 PM »
@JMD scraping works well with launchobx and attract mode, so I don't think it's a network issue.

@mgalaxy of course I'm available to try a debug build :-)

mgalaxy

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
    • Email
Re: Scraper issue
« Reply #12 on: January 27, 2019, 10:08:45 AM »
OK, I got it...seems to be a chance in the way the info is sent from the database server...I check

mgalaxy

  • Administrator
  • Hero Member
  • *****
  • Posts: 1048
    • Email
Re: Scraper issue
« Reply #11 on: January 27, 2019, 09:49:07 AM »
VONBLUBBA,
I'm extensively checking for your problem...but can't reproduce it here.
I am writing a special version to debug this particular problem and will propose it for you to download and test if you agree!?