Re: [xmltv-users] UK Atlas load/usage warning email
Robert Eden <[email protected]>
| Newsgroups | gmane.comp.tv.xmltv.devel |
|---|---|
| Message-ID | <[email protected]> |
On 5/12/2016 2:43 AM, [email protected] wrote: > On Wed, 11 May 2016 15:38:56 -0500, Robert Eden wrote: >> The way I read the last email from Atlas is a single request can't cover >> over 24 hours. Having written a back-end server, I can understand why. >> (it's a very big "in" close to get details) >> >> The solution is to update the grabber to break up requests into smaller >> chunks..... 4 hours, 12hours, etc. It's not that the grabber is >> requesting 7 days... it's that the grabber is requesting 7 days *in a >> single call* >> >> Robert > On the face of it that could appear to be a solution, however I believe there's more to it than that. Already the grabber fetches each channel separately, so for a lineup of say 150 channels that's 150 API calls. To subdivide that into one call per 24 hours means a 7-day fetch becomes over 1,000 calls. I seriously doubt that is a lower load on the server, or is what MB want. And I can see the next pronouncement being "max 50 calls per person per day". > > If Metabroadcast will unequivocally state that getting 7 days' worth of data as 7 consecutive API calls is allowed, then ok. Otherwise I believe it will be an ultimately fruitless exercise. > > (Note - the Atlas grabber already has the option to fetch by --hours not just --days ) The Schedules Direct DataDirect backend outputs normalized data. There's a schedule section, program section, credits section, etc. Does Atlas output XMLTV directly? tv_grab_na_dd uses a *lot* of memory caching all the data so it can generated non-normalized XMLTV format data. If that approach was done server-side, I can see it causing lots of memory issues. by breaking things down into less-than-a-day chunks, Atlas hopes to reduce memory usage in their server. I suspect tv_grab_uk_atlas is a significant number of their users... it would have been nice if they contacted us to resolve issues rather than send boilerplate blasts to users. Robert ------------------------------------------------------------------------------ Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j