Should wurfl.zip extract every time??

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Should wurfl.zip extract every time??

Postby keith » Thu May 30, 2013 2:30 pm

I'm testing out the example on http://wurfl.sourceforge.net/php_index.php

Are we supposed to remove the following line after the initial execution?
...
$wurflConfig->wurflFile($resourcesDir.'/wurfl.zip');
...

It seems that the files in the persistence directory are overwritten each time and this process is taking upwards of 30 seconds. It's not clear in the docs I have read so far why this would be happening or if maybe I should not point to this file again.

kamermans
Posts: 393
Joined: Mon Jun 06, 2011 9:50 am

Re: Should wurfl.zip extract every time??

Postby kamermans » Thu May 30, 2013 4:37 pm

You don't need to remove that line. The WURFL data should only be reloaded if the wurfl.zip file has changed or if the persistence data is incomplete. This behaviour is enabled with the auto_reload option. If the data is being reloaded all the time, there must be something wrong, but I would check first to see if there is some reason that the wurfl.xml file's mtime is changing or something similar. In the meantime, you can set auto_reload to false to prevent the reloading.
Thanks,

Steve Kamerman
ScientiaMobile

Make sure you check out our WURFL Cloud, WURFL InSight and WURFL InFuze products!

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Fri May 31, 2013 1:31 pm

Hey, thanks for the reply.

I think I tracked it down. It looks like the guy who was working on this the other day set the .zip path to an older version of this file.

My config was pointing to:

resource_file/wurfl-2.3.3.zip
1289038 May 29 11:16 wurfl-2.3.3.zip
which contains:
20964986 11-08-12 12:27 wurfl.xml

Now if I point it to the following file, the extraction only happens the first time (good, this is what we want):

storage/wurfl.zip
1312502 May 29 11:16 wurfl.zip

which contains
21302886 01-06-13 20:53 wurfl.xml

Note: Neither zip or xml files were changed at any point. So why would the config trigger the extraction when pointing to the older file? Is something about the file stored elsewhere?

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Fri May 31, 2013 3:26 pm

UPDATE:

It seems that the .zip extracts every time if I am using memcache. Anyone know anything about this?

$wurflConfig->cache('memcache', array('host' => my.host.foo));

If I switch this back to 'file' the .zip does not extract every time.

kamermans
Posts: 393
Joined: Mon Jun 06, 2011 9:50 am

Re: Should wurfl.zip extract every time??

Postby kamermans » Fri May 31, 2013 3:54 pm

Thanks for the update. Can you post your complete configuration please? I will try to reproduce it on our side, as this is not the expected behavior.
Thanks,

Steve Kamerman
ScientiaMobile

Make sure you check out our WURFL Cloud, WURFL InSight and WURFL InFuze products!

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Mon Jun 03, 2013 10:37 am

Hey kamermans , here's pretty much what I'm doing minus actual file paths and memcache host. It seems that something si wrong with the memcache implementation and I'm not really sure how to troubleshoot.

Code: Select all


require_once 'wurfl/Application.php';

class MYWURFLFactory extends WURFL_WURFLManagerFactory{

  public function __construct() {
  
    $persistenceDir = 'storage/persistence';
    $cacheDir = 'storage/cache';
      
    $wurflConfig = new WURFL_Configuration_InMemoryConfig();
    $wurflConfig->wurflFile(storage/wurfl.zip');
    $wurflConfig->matchMode('accuracy');
    
    $wurflConfig->persistence('file', array('dir' => $persistenceDir));
    
    
    // when using 'memcache' this has no affect
    $wurflConfig->allowReload(false);
    
    // when using 'file' everything works as expected
    // $wurflConfig->persistence('file', array('dir' => $persistenceDir));    
    
    // when using 'memcache' the contents of the persistence dir are rewritten every time
    $wurflConfig->cache('memcache', array('host' => 'my.memchache.host.amazonaws.com'));
    
    parent::__construct($wurflConfig);
  
  }

}


kamermans
Posts: 393
Joined: Mon Jun 06, 2011 9:50 am

Re: Should wurfl.zip extract every time??

Postby kamermans » Mon Jun 03, 2013 2:53 pm

Ok, thanks for the config, I'll take a look at it. Are you using Amazon ElastiCache, or an actual Memcached installation on an Amazon VM, and is the Memcached host the same as your webserver, or is it a different host altogether?
Thanks,

Steve Kamerman
ScientiaMobile

Make sure you check out our WURFL Cloud, WURFL InSight and WURFL InFuze products!

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Mon Jun 03, 2013 3:49 pm

Thanks Steve,

In our developer environment we use Amazon ElastiCache. In production we use our own cluster of machines with memcache installed on them.

Keith

kamermans
Posts: 393
Joined: Mon Jun 06, 2011 9:50 am

Re: Should wurfl.zip extract every time??

Postby kamermans » Tue Jun 04, 2013 11:14 am

Hi Keith,

Sorry about the delay. I've found an issue with the Memcache adapter and I'm working on a solution.
Thanks,

Steve Kamerman
ScientiaMobile

Make sure you check out our WURFL Cloud, WURFL InSight and WURFL InFuze products!

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Tue Jun 04, 2013 11:40 am

Thanks! For now I will stick to 'file' until a solution is available.

kamermans
Posts: 393
Joined: Mon Jun 06, 2011 9:50 am

Re: Should wurfl.zip extract every time??

Postby kamermans » Tue Jun 04, 2013 4:33 pm

I've got a fix ready for you - I will PM it to your account later tonight once I've created the package.
Thanks,

Steve Kamerman
ScientiaMobile

Make sure you check out our WURFL Cloud, WURFL InSight and WURFL InFuze products!

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Wed Jun 05, 2013 9:52 am

Thanks Steve! I will test it out today and let you know.

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Wed Jun 05, 2013 1:33 pm

Steve,

I did some tests and the long write/extraction seems to be gone now even in cache 'file' mode. I've cleared the cache and persistence directories before each test. Is this expected to bypass entirely with memcache? Is data supposed to write to 'cache' or 'persistence' even in memcache mode?


Also, I am starting to see these Fatal errors on some user-agents. Prior to this update this was not happening:

Fatal error: Uncaught exception 'Exception' with message 'There is no device with ID [apple_iphone_coremedia_ver1] in the loaded WURFL Data' in /path/to/wurfl/wurfl-php-1.4.4.1/WURFL/CustomDeviceRepository.php:104 Stack trace: #0 /path/to/wurfl/wurfl-php-1.4.4.1/WURFL/CustomDeviceRepository.php(207): WURFL_CustomDeviceRepository->getDevice('apple_iphone_co...') #1 /path/to/wurfl/wurfl-php-1.4.4.1/WURFL/WURFLService.php(137): WURFL_CustomDeviceRepository->getDeviceHierarchy('apple_iphone_co...') #2 /path/to/wurfl/wurfl-php-1.4.4.1/WURFL/WURFLService.php(62): WURFL_WURFLService->getWrappedDevice('apple_iphone_co...', Object(WURFL_Request_MatchInfo)) #3 /path/to/wurfl/wurfl-php-1.4.4.1/WURFL/WURFLManager.php(94): WURFL_WURFLService->getDeviceForRequest(Object(WURFL_Request_GenericRequest)) #4 /path/to/wurfl/wurfl-php-1.4.4.1/WURFL/WURFLManager.php( in /path/to/wurfl/wurfl-php-1.4.4.1/WURFL/CustomDeviceRepository.php on line 104

kamermans
Posts: 393
Joined: Mon Jun 06, 2011 9:50 am

Re: Should wurfl.zip extract every time??

Postby kamermans » Thu Jun 06, 2013 1:09 pm

With memcache cache in front of filesystem persistence, if memcache can completely serve the request, there may be no need to access the filesystem, although the cache will expire eventually and the persistence information will be read from disk and re-cached.

With regards to the apple_iphone_coremedia_ver1 issue, there is something strange going on here, because this ID has been in WURFL for a long time. Were you using memcache for persistence? That is generally a bad idea since the cache data can push persistent data out of memcache, then the WURFL API will try to lookup a device that does not exist.
Thanks,

Steve Kamerman
ScientiaMobile

Make sure you check out our WURFL Cloud, WURFL InSight and WURFL InFuze products!

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Thu Jun 06, 2013 4:56 pm

Hey Steve,

I just rebooted the dev memcache instance. Then made an API call, the first one takes a while as it extracts the persistence data. No errors and everything worked fine on subsequent calls.

So as as test I decided to clear out the persistence and cache directories on the disk and this is when I start getting those random fatal errors (while still using memcache). I guess the only way to regenerate this data is to set cache back to 'file' and I'm guessing that there's no reason I should be clearing those directories in the first place. I think I'm probably good now.

kamermans
Posts: 393
Joined: Mon Jun 06, 2011 9:50 am

Re: Should wurfl.zip extract every time??

Postby kamermans » Thu Jun 06, 2013 6:45 pm

Ok, that makes sense. The API checks for the existence of one specific file in the persistence directory and assumes that the rest are also present. Deleting files in the persistence directory will certainly cause problems.

In order to reload the data, you should ensure auto_reload is enabled, then either replace your wurfl.zip/wurfl.xml file with a different version, or just touch it. The persistence and cache data will be regenerated the during the next request.
Thanks,

Steve Kamerman
ScientiaMobile

Make sure you check out our WURFL Cloud, WURFL InSight and WURFL InFuze products!

keith
Posts: 13
Joined: Thu May 30, 2013 2:24 pm

Re: Should wurfl.zip extract every time??

Postby keith » Fri Jun 07, 2013 11:41 am

Ah, good idea about touch. That would be a really easy way to force testing. Thanks again for the help!


Who is online

Users browsing this forum: No registered users and 3 guests