Help implementing WURFL InFuze for Varnish-Cache

Questions about our enterprise "in the network" WURFL components
kirsten_sekhaolelo
Posts: 18
Joined: Thu May 29, 2014 10:38 am

Help implementing WURFL InFuze for Varnish-Cache

Postby kirsten_sekhaolelo » Tue Jun 17, 2014 4:16 am

Hi
My oganisation is the holder of WURFL InFuze for Varnish-Cache lincense.
Our servers are running Ubuntu 14.04 with varnish version 3.0.5..
We are running WordPress-powered news site.

We want to utilise WURFL InFuze for Varnish-Cache to serve different themes to different device types.

We want to segregate the devices hitting our servers into 3 classes:
1. Class A -> PCs and tablets;
2. Class B -> high-end mobile devices, and
3. Class C -> low-end devices/feature phones.

We have two themes/Designs on our WordPress site: Responsive theme and Mobi themes.

We want to serve the different themes to different device Class categories:
• Class A and Class B devices will get the Responsive theme.
• Class C will get the Mobi theme.

A good example of high-end and low-end devices would with Blackberry phones. We have:
• BlackBerry PlayBook -> tablet, BlackBerry PlayBook, 1024 x 600 resolution
• BlackBerry Z30 -> BlackBerry 10 operating system, 1280 x 720 resolution, full touch device
• BlackBerry Q10 -> BlackBerry 10 operating system, 720 x 720 resolution, keyboard & touch
• BlackBerry Z10 -> BlackBerry 10 operating system, 1280 x 768 resolution, full touch device
These are our high-end devices (Class B) and will get the Responsive theme.

Then we have:
• BlackBerry 9720 -> BlackBerry 7 OS , 480 x 360 resolution, keyboard & touch
• BlackBerry Bold -> BlackBerry 7 OS, 640 x 480) screen resolution, keyboard & touch
• BlackBerry Curve -> BlackBerry 7 OS, 480 x 360 screen resolution
• And all the blackberry phones running Blackberry 7 OS and downwards with screen resolution width of 320 and below.
These are our low-end devices (Class C) and will get the Mobi theme.

The WURLF capabilities that we are looking use include:
brand_name, is_wireless_device, device_os, device_os_version, pointing_method, is_smartphone, is_table, resolution_width, resolution_height, mobile_browser, and mobile_browser_version.

A combination of whether a device is wireless device, whether it is tablet, whether it touchscreen, screen resolution and combination of operating system and their versions, and browsers and their version, should be enough to enable us to segregate the devices.

I am PHP developer with very limited knowledge of VCL. In the past few weeks I have undertaken learning the language.
We need help implementing the WURFL InFuze for Varnish-Cache to achieve the goals mentioned above.

Elliotfehr

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby Elliotfehr » Tue Jun 17, 2014 9:44 am

Hello,

From what you have described, it is definitely possible to differentiate between these devices using WURFL. Here is a comprehensive guide on how to get started using the Varnish-Cache InFuze module. Feel free to let us know if you have any questions that are not covered in this guide.

Thank you,

Elliot

kirsten_sekhaolelo
Posts: 18
Joined: Thu May 29, 2014 10:38 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby kirsten_sekhaolelo » Fri Jun 20, 2014 6:11 am

Thanks for the response Elliotfehr.

I took time to go through the user guide and also went through the post by Luca Passani here(http://www.scientiamobile.com/page/http ... ing-header).

I will not focus on the installation part. Our Linux engineers will take care of that part.

From example varnish.sample.vcl and post by Luca Passani I came up with this VCL configuration file.

Code: Select all

##########################################################################
#############Beginning on my configuration file:#################################
import wurfl;
import std;

backend apache {
	.host = "127.0.0.1";
	.port = "8080";
}

sub vcl_init {
	wurfl.set_root("/usr/share/wurfl/wurfl-eval.xml");
	#wurfl.add_patch("...");

	wurfl.set_engine_target_high_performance();
	#wurfl.set_engine_target_high_accuracy();

	#wurfl.set_cache_provider_none();
	#wurfl.set_cache_provider_lru(10000);
	wurfl.set_cache_provider_double_lru(10000,3000);

	# Mandatory Capabilities: You MUST leave the following capability requests
	wurfl.add_requested_capability("device_os");
	wurfl.add_requested_capability("device_os_version");
	wurfl.add_requested_capability("is_tablet");
	wurfl.add_requested_capability("is_wireless_device");
	wurfl.add_requested_capability("pointing_method");
	wurfl.add_requested_capability("preferred_markup");
	wurfl.add_requested_capability("resolution_height");
	wurfl.add_requested_capability("resolution_width");
	wurfl.add_requested_capability("ux_full_desktop");
	wurfl.add_requested_capability("xhtml_support_level");

	# User required capabilities
	wurfl.add_requested_capability("brand_name");
	wurfl.add_requested_capability("device_os");
	wurfl.add_requested_capability("device_os_version");
	wurfl.add_requested_capability("is_tablet");
	wurfl.add_requested_capability("is_wireless_device");
	wurfl.add_requested_capability("pointing_method");
	wurfl.add_requested_capability("preferred_markup");
	wurfl.add_requested_capability("resolution_height");
	wurfl.add_requested_capability("resolution_width");
	wurfl.add_requested_capability("mobile_browser");
	wurfl.add_requested_capability("mobile_browser_version");
	wurfl.add_requested_capability("ajax_support_javascript");
	wurfl.add_requested_capability("ajax_manipulate_css");
	wurfl.add_requested_capability("ajax_support_getelementbyid");
	wurfl.add_requested_capability("ajax_support_inner_html");

	wurfl.load();

	if (wurfl.error()) {
			std.syslog(0, wurfl.error());
			panic wurfl.error();
	}
}

sub vcl_recv {
    set req.http.X-Wurfl-Id = wurfl.get_device_id();
    set req.http.X-Wurfl-Cap-Brand-Name = wurfl.get_capability(“brand_name”);
    set req.http.X-Wurfl-Cap-Device-Os = wurfl.get_capability(“device_os”);
    set req.http.X-Wurfl-Cap-Device-Os-Version = wurfl.get_capability(“device_os_version”);
    set req.http.X-Wurfl-Cap-Is-Tablet = wurfl.get_capability(“is_tablet”);
    set req.http.X-Wurfl-Cap-Is-Wireless-Device = wurfl.get_capability(“is_wireless_device”);
    set req.http.X-Wurfl-Cap-Pointing-Method = wurfl.get_capability(“pointing_method”);
     set req.http.X-Wurfl-Cap-Resolution-Width = wurfl.get_capability(“resolution_width”);
    set req.http.X-Wurfl-Cap-Resolution-Height = wurfl.get_capability(“resolution_height”);
    set req.http.X-Wurfl-Cap-Mobile-Browser = wurfl.get_capability(“mobile_browser”);
    set req.http.X-Wurfl-Cap-Mobile-Browser_version = wurfl.get_capability(“mobile_browser_version”);
    set req.http.X-Wurfl-Cap-Ajax-Support-Javascript = wurfl.get_capability(“ajax_support_javascript”);
    set req.http.X-Wurfl-Cap-Ajax-Manipulate-Css = wurfl.get_capability(“ajax_manipulate_css”);
    set req.http.X-Wurfl-Cap-Ajax-Support-Getelementbyid = wurfl.get_capability(“ajax_support_getelementbyid”);
    set req.http.X-Wurfl-Cap-Ajax-Support-Inner-Html = wurfl.get_capability(“ajax_support_inner_html”);
    
    #Virtual Capabilities
    set req.http.X-Wurfl-Cap-Is-Smartphone = wurfl.get_virtual_capability("is_smartphone");
    set req.http.X-Wurfl-Cap-Is-Touchscreen = wurfl.get_virtual_capability("is_touchscreen");
    
    if (wurfl.error()) {
		std.syslog(0, wurfl.error());
		panic wurfl.error();
	}
}

sub vcl_hash {
	hash_data(req.url+req.http.host+wurfl.get_device_id());
	return (hash);
}


#####################End of configuration file##############################
Please correct me if and where I am wrong and also give advice.

What I am trying to achieve here is firstly to add the WURFL capabilities in the vcl_init sub.
In the vcl_recv sub I am adding WURFL capabilities as headers in the HTTP specs so that I can be able to grab them in my PHP script and evaluate the nature of the device and direct users to the relevant design/theme.
In the vcl_hash sub I am using the hash_data() function and pass the requested URL + the Device ID matched by WURFL as parameter as done in example 3 of the user guide. The reason I am doing that is because in my PHP script I will direct a certain group of devices to a certain theme/design based on the WURLF capabilities passed as variables in the HTTP headers and once the object has been cached using the hash value made of the requested URL with the Device ID, I am hoping devices of the same type will then access the same cached object, because they have the same device ID and access the same URL. The hash request will match and the object will be served from cache.

Thanks in advance for your input.

Kirsten

support-all
Posts: 32
Joined: Mon Oct 08, 2012 2:47 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby support-all » Fri Jun 20, 2014 8:13 am

Hi Kirsten,

you're taking the right approach with your vcl, here you are a few advices:

-) Since you're looking for detecting both desktop and mobile devices I would recommend to set the high_accuracy mode instead of high_performance. In this case WURFL will be way more accurate in detecting both device categories and give you less generic results, with a little computation overhead.
-) I've noticed that you've requested the same capability multiple times in vcl_init subroutine, for example "device_os" and "is_tablet". There's no need to request the same capability more than once, so you can safely remove those duplicated lines.
-) Here http://www.scientiamobile.com/wurflCapability you can find all the WURFL supported capabilities and virtual capabilities. If you browse the virtual capability section you'll see that "is_full_desktop" and "advertised_device_os" could also suit your needs, considering the Class A,B,C distinction you're going to make in the underlying php logic.
-) Finally, your vcl_hash subroutine looks good to me, Varnish will hash the concatenation of these three strings and that will be your hashed object to check the next time Varnish receives a request.

Varnish also have vcl_miss and vcl_hit subroutines, useful to check whether Varnish missed or hit a specific request in cache. Take a look at https://www.varnish-cache.org/docs/3.0/ ... #functions available Varnish 3.0.5 subroutines for further information.

As an extra advice consider that you're able to define your own XML patch file, which can be subsequently loaded into WURFL using the add_patch function.
In this XML patch you can define your own devices (using the same tags as we used in the main wurfl-eval.xml) and, for each one of them, the device-specific capabilities values. Also note that every device must at least fallback to our "generic" device defined in wurfl-eval.xml, otherwise WURFL will not recognise your device to be a valid WURFL device. However, for each custom defined device, you can override every capability defined for the "generic" device.

(Right now the upload attachment function in our forum is causing some troubles, so I can email an XML patch file example if you want)

Kind regards,
Andrea

kirsten_sekhaolelo
Posts: 18
Joined: Thu May 29, 2014 10:38 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby kirsten_sekhaolelo » Mon Jun 23, 2014 3:20 am

Hi Andrea

I have considered and followed your advice.

Below is the updated VCL config file.

I have addressed the following advices:
1. I have replaced high_performance mode with high_accuracy mode
2. I have removed duplicated lines in vcl_init subroutine
3. As per your advice I have added "is_full_desktop" and "advertised_device_os" virtual capabilities to use in my PHP script in vcl_recv subroutines
4. Implemented the vcl_miss and vcl_hit subroutines
5. You can email me an XML patch file example. But the reason we opted for WURFL was so that we won’t have to worry about defining devices as they are released. We do not have the capacity to do and as such we will rely 100% on the WURFL xml file.
##########################################################################
#############Beginning on my configuration file:#################################
import wurfl;
import std;

backend apache {
.host = "127.0.0.1";
.port = "8080";
}

sub vcl_init {
wurfl.set_root("/usr/share/wurfl/wurfl-eval.xml");
#wurfl.add_patch("...");

#wurfl.set_engine_target_high_performance();
wurfl.set_engine_target_high_accuracy();

#wurfl.set_cache_provider_none();
#wurfl.set_cache_provider_lru(10000);
wurfl.set_cache_provider_double_lru(10000,3000);

# Mandatory Capabilities: You MUST leave the following capability requests
wurfl.add_requested_capability("device_os");
wurfl.add_requested_capability("device_os_version");
wurfl.add_requested_capability("is_tablet");
wurfl.add_requested_capability("is_wireless_device");
wurfl.add_requested_capability("pointing_method");
wurfl.add_requested_capability("preferred_markup");
wurfl.add_requested_capability("resolution_height");
wurfl.add_requested_capability("resolution_width");
wurfl.add_requested_capability("ux_full_desktop");
wurfl.add_requested_capability("xhtml_support_level");

# User required capabilities
wurfl.add_requested_capability("brand_name");
wurfl.add_requested_capability("mobile_browser");
wurfl.add_requested_capability("mobile_browser_version");
wurfl.add_requested_capability("ajax_support_javascript");
wurfl.add_requested_capability("ajax_manipulate_css");
wurfl.add_requested_capability("ajax_support_getelementbyid");
wurfl.add_requested_capability("ajax_support_inner_html");

wurfl.load();

if (wurfl.error()) {
std.syslog(0, wurfl.error());
panic wurfl.error();
}
}

sub vcl_hit {
return (deliver);
}

sub vcl_miss {
return (fetch);
}

sub vcl_recv {
set req.http.X-Wurfl-Id = wurfl.get_device_id();
set req.http.X-Wurfl-Cap-Brand-Name = wurfl.get_capability(“brand_name”);
set req.http.X-Wurfl-Cap-Device-Os = wurfl.get_capability(“device_os”);
set req.http.X-Wurfl-Cap-Device-Os-Version = wurfl.get_capability(“device_os_version”);
set req.http.X-Wurfl-Cap-Is-Tablet = wurfl.get_capability(“is_tablet”);
set req.http.X-Wurfl-Cap-Is-Wireless-Device = wurfl.get_capability(“is_wireless_device”);
set req.http.X-Wurfl-Cap-Pointing-Method = wurfl.get_capability(“pointing_method”);
set req.http.X-Wurfl-Cap-Resolution-Width = wurfl.get_capability(“resolution_width”);
set req.http.X-Wurfl-Cap-Resolution-Height = wurfl.get_capability(“resolution_height”);
set req.http.X-Wurfl-Cap-Mobile-Browser = wurfl.get_capability(“mobile_browser”);
set req.http.X-Wurfl-Cap-Mobile-Browser_version = wurfl.get_capability(“mobile_browser_version”);
set req.http.X-Wurfl-Cap-Ajax-Support-Javascript = wurfl.get_capability(“ajax_support_javascript”);
set req.http.X-Wurfl-Cap-Ajax-Manipulate-Css = wurfl.get_capability(“ajax_manipulate_css”);
set req.http.X-Wurfl-Cap-Ajax-Support-Getelementbyid = wurfl.get_capability(“ajax_support_getelementbyid”);
set req.http.X-Wurfl-Cap-Ajax-Support-Inner-Html = wurfl.get_capability(“ajax_support_inner_html”);

#Virtual Capabilities
set req.http.X-Wurfl-Cap-Is-Smartphone = wurfl.get_virtual_capability("is_smartphone");
set req.http.X-Wurfl-Cap-Is-Touchscreen = wurfl.get_virtual_capability("is_touchscreen");
set req.http.X-Wurfl-Cap-Is-Full-Desktop = wurfl.get_virtual_capability("is_full_desktop");
set req.http.X-Wurfl-Cap-Aadvertised-Device-Os = wurfl.get_virtual_capability("advertised_device_os");
set req.http.X-Wurfl-Cap-advertised-Device-Os-Version = wurfl.get_virtual_capability("advertised_device_os_version");

if (wurfl.error()) {
std.syslog(0, wurfl.error());
panic wurfl.error();
}
}

sub vcl_hash {
hash_data(req.url+req.http.host+wurfl.get_device_id());
return (hash);
}


#####################End of configuration file##############################
Again your input will be highly appreciated.

One question: do I have to implement vcl_pipe, vcl_pass, vcl_deliver, vcl_fetch subroutines?

Thanks guys.

Kirsten

support-all
Posts: 32
Joined: Mon Oct 08, 2012 2:47 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby support-all » Mon Jun 23, 2014 9:48 am

Hi Kirsten,

It's not strictly necessary to implement all vcl subroutines, since they act as callbacks for the Varnish user which needs to intervene in a particular flow prior to cease the control to Varnish Cache again (for example vcl_pass is used when Varnish is set to "pass" mode, and in this case Varnish will pass the request directly to the backend, thus not using his cache).

-) Keep "vcl_hit" if you need to do actions triggered when a Varnish cache hit happens.
-) Keep "vcl_miss" if you need to do actions triggered when a Varnish cache miss happens (for example, log a warning message stating that a cache miss had occurred).
-) Keep the "vcl_hash" you wrote in the vcl.

The other subroutines are to be used in special cases, and I encourage you to take a look at the Varnish docs https://www.varnish-cache.org/docs/trun ... -subs.html to learn about advanced Varnish functionalities and modes.

Since you'll rely completely on the wurfl.xml, a possible way to categorise the devices you're interested in is the following (manual approach, instead of a wurfl patch file):

1) Use the vcl you pasted, with the modifications I mentioned above.
2) Start varnish
3) Use "curl" via command line with a specific user agent, representing the device you want to categorise.

For example, let's say you need to know how WURFL handles Blackberry Bold 9900, having user agent "Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en-US) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.0.0 Mobile Safari/534.11+".
Assuming that you've started Varnish on 127.0.0.1:6081 then type the following command:

# curl -H 'User-Agent: Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en-US) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.0.0 Mobile Safari/534.11+' 127.0.0.1:6081

If you print the headers in the request received by your web server you'll get all infos you requested in your vcl_recv subroutine, hence letting you categorise this device based on its device_id, for example.
Clearly the "curl" query have to be repeated once for every user agent you want to categorise.

Let us know if there's something else we can do to help you.

Best Regards,
Andrea

kirsten_sekhaolelo
Posts: 18
Joined: Thu May 29, 2014 10:38 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby kirsten_sekhaolelo » Mon Jun 23, 2014 10:09 am

Hi Andrea

Thanks for getting back to me.

Please send me the XML patch file example. I think I would go with the patch file rather than the manual approach you stated.

Also please explain to me why I need this patch file if have the WURLF xml file and how exactly does it work.

Thanks.

Regards.

Kirsten

support-all
Posts: 32
Joined: Mon Oct 08, 2012 2:47 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby support-all » Mon Jun 23, 2014 10:18 am

Ok, I'll send you a patch file example, along with an explanation on how to use the patch with WURFL.

Thanks,
Andrea

kirsten_sekhaolelo
Posts: 18
Joined: Thu May 29, 2014 10:38 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby kirsten_sekhaolelo » Sun Jun 29, 2014 2:07 pm

Hi

Where do I find libwurfl rpm package?

Regards,

Kirsten

kirsten_sekhaolelo
Posts: 18
Joined: Thu May 29, 2014 10:38 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby kirsten_sekhaolelo » Sun Jun 29, 2014 2:15 pm

I found them. Thanks for your help.

Regards,

Kirsten

kirsten_sekhaolelo
Posts: 18
Joined: Thu May 29, 2014 10:38 am

Re: Help implementing WURFL InFuze for Varnish-Cache

Postby kirsten_sekhaolelo » Mon Jun 30, 2014 9:31 am

Hi Andrea and Co

I finally got the detection working. Thanks for all your assistance and patience.

It does exactly what I wanted it to do.

Thanks again.

Best regards,

Kirsten


Who is online

Users browsing this forum: No registered users and 3 guests