DataUnlocker

General information

What is DataUnlocker

DataUnlocker is a Software-as-a-Service platform helping web applications to work as intended, despite any third-party software intrusion on client devices or network problems. A typical use case for installing DataUnlocker in a web application is unblocking previously blocked browser requests, caused for example by ad blockers, network policies or other types of content blocking software.

DataUnlocker Concepts

Simply put, DataUnlocker is a (smart) reverse proxy service, which proxies blocked network requests from web applications, under the hood. Thus, it makes a target server get previously blocked data, and a web application to receive a desired response, like it wasn't blocked. From the web application perspective, DataUnlocker requires a single-time installation only, with no changes to existing code.

The main benefit that DataUnlocker brings to web applications is the recovery of client-side analytics collection (which, as you may have noticed, we use across our titles). However, because DataUnlocker is an invisible (smart) proxy by its nature, here are other things it can do:

• Perhaps, increase your ads conversion (while this is not the intended use of DataUnlocker). While ads are usually blocked by their HTML code (which DataUnlocker has nothing to do with), some may start being visible, as now blocked requests to ad services can also be proxied. Need to proxy an iframe content? You can request this feature, as well as other features.
• Easy but advanced control over exactly what DataUnlocker will proxy and how.
• Out-of-the-box CORS support, meaning that you will be able to display resources from other websites or APIs that have CORS, without any code changes (coming soon).
• Different features of proxied services are included: for example, in Google Analytics, you can tell DataUnlocker to mark recovered hits as "unblocked" by creating a custom dimension. Later, you can use Google Analytics to analyze how many requests or users were recovered after ad blockers.

How DataUnlocker works

We have briefly explained how DataUnlocker bypasses content blockers on our landing page; let us put it here as well:

Put simply, DataUnlocker asks you to create a new unique "URL prefix", which is either a DNS record or a path prefix at your domain, which is obviously not present in any of the content filtering lists. DataUnlocker will proxy blocked requests using this unique prefix.

Because proxying requests to some third-party services may have unwanted side effects, DataUnlocker proxies only blocked traffic to not to break any existing functionality of your web application or third-party services it is connected to. To achieve this, DataUnlocker's script monkey-patches certain browser APIs, enabling an invisible proxy layer to its proxy servers.

The following diagram demonstrates a sequence of actions for successful and blocked requests, when DataUnlocker script is installed. Please note that by default DataUnlocker tries to proxy all failed requests regardless of their status, including requests returning 4xx or 5xx HTTP statuses in the response. To change this behavior, see configuring what to proxy.

Behind the scenes, DataUnlocker performs many additional "magic" to make it work, which includes, but is not limited to:

• Encoding and decoding client request and response data.
• Transforming the client's request so that the target third-party server understands that it was proxied.
• Monitoring web applications and content filtering lists to ensure web apps stay unblocked.
• Any additional work required to keep its web applications unblocked, such as hot-swapping DataUnlocker servers' IP addresses if they end up in any of the blacklists.
• Proxying the request further to its correct geolocation in 120+ countries (not yet implemented; let us know if you need it).
• Dynamic DNS management to stay unblocked no matter what (not yet implemented; let us know if you need it).

In the unlikely event of content blockers becoming smarter to detect DataUnlocker's job, DataUnlocker will notify its customers with the given impact and possible update steps for their configuration. Blocking DataUnlocker is not feasible from the content blockers' perspective, as they may end up either breaking a lot of websites without a reason or blacklisting a good part of the Internet.

The only thing DataUnlocker cannot guarantee is dataunlocker.com domain not ending up in some of the blacklists. This will not impact DataUnlocker's customers in any way (because they all have a unique setup to their own domain names). As for DataUnlocker's website, we expect our customers to be ready for adding an exception for dataunlocker.com in their ad blocker, just like they already are with f.e. Google Analytics.

Installation process

In DataUnlocker's terms, websites or web applications are called Properties. When adding a new property in DataUnlocker Admin, you will be asked to complete a 3-step property onboarding process. This process takes around 5-10 minutes if you have full access to your website and a domain. You can also watch a video tutorial explaining how to install DataUnlocker.

Note that you will be able to come back to any of the installation steps later.

Choosing a proxy routing option

In the first step, you will be asked to choose a routing option, which can be either:

• DNS-based
• path-based

You may choose a DNS-based option if you have an access to your domain, as it is the easiest option available. The second option requires you to set up a reverse proxy on your side, which will talk to our (smart) proxy. If you wonder what exactly choosing one of these options means, here are some considerations:

• The path-based routing option can be used when your (website hosting) provider doesn't give you access to control your domain.
• The path-based routing option can bring you a benefit of same-origin proxied requests, however, DataUnlocker will soon support CORS tuning anyway, which will make CORS also possible for the DNS option. Note that all third-party solutions like Google Analytics use cross-origin requests by their nature, hence CORS is not an issue even when proxying requests via another domain.

After choosing your preferred option, the setup will suggest a DNS configuration (or a reverse proxy configuration if you have chosen a path-based method) for your website. Follow the suggested steps in the UI.

Installing a script

After completing the first setup step, you will be given a script to install in your web application. Note that changing the routing option will require updating a script, as it is based on the chosen routing option. Follow the instructions of the script installation user interface, and note the next:

• It is important to place the script directly to your web application's HTML code, before all other scripts, as otherwise it may not be effective for all web application users. This is a synchronous script, which monkey-patches mission-critical browser's APIs (namely, XMLHttpRequest, fetch, navigator.sendBeacon, document.createElement, starts to listen on <script> and <img> "error" events, etc). It also hooks into other scripts already present in the HTML code, so that you don't need to change your original code. Even more than that, XMLHttpRequest and fetch will proxy failed requests under-the-hood if it is configured in DataUnlocker Admin, so that you won't f.e. receive an "error" event up until the proxy attempt errors.
• In case you are having issues after installing DataUnlocker's script (which is not expected once DataUnlocker reaches general availability), you can use an additional settings icon to get the debug version of the script.
• You won't see DataUnlocker's script in the DOM (f.e. in the browser's developer tools) because it removes itself.

Despite DataUnlocker has integration tests for many scenarios, before it is generally available, it is recommended to install the script in your development environment first. Just inserting the script's code, even when DataUnlocker is turned off, should be fine to verify that monkey-patching doesn't break your web application. Please report any unexpected behavior, and also make sure you have a debug version of the script installed for us to take a look.

Using continuous integration to keep the script updated

By design, DataUnlocker's script must be inserted directly to the HTML code of your pages, which prevents you from keeping this script up to date, or otherwise adds some manual work to update the script once new version is rolled out. DataUnlocker will remind you over email to update the script only in case it is absolutely necessary to do so, however, there is another approach to always have the latest script installed.

DataUnlocker exposes the following public API endpoint for injecting its script into HTML code:

• https://api.dataunlocker.com/properties/{property-id}/scripts/latest/inject

Thus, you can use curl CLI utility to inject DataUnlocker's script to any HTML page, for instance:

• curl -F html=@index.html 'https://api.dataunlocker.com/properties/{property-id}/scripts/latest/inject' > index.html

This command will take existing index.html file and replace it with the same HTML file but with the latest DataUnlocker script installed. You can use this injection with your production HTML file to keep DataUnlocker's script up to date.

This endpoint is also smart enough to recognize DataUnlocker scripts which may already be present in the HTML, even if they are formatted or changed (but given they are still executable). This feature allows you to combine the static script version installation with dynamic script injection, which guarantees that the script will be always served (even in case when injection fails for whatever reason).

Processing the entire HTML file also leaves a chance for DataUnlocker to do more processing in the future, like encoding already-present analytics scripts to prevent some ad blockers from recognizing them by reading the HTML code. However, encoding is not happening at the moment.

Verifying the installation

Once the proxy path option is chosen and the script is installed, DataUnlocker will assess your installation and output any problems it may have, if there are any. Follow the instructions to complete the health check. You may need to retry the health check a couple of times before it becomes all-green.

Click on failed health assessments to see why it failed. If the health-check either finds a script matching property or the correct DNS record, your property will become "verified" and you will see more pages available for you in the left navigation menu.

Configuration

Once your health check passes, you can start to configure what and how will be proxied. This process is optional, however, it is recommended to turn the "Proxy services which are not configured" option off after the initial configuration. Read more below.

Configuring what to proxy

By default, DataUnlocker's proxy is turned off. After the successful health check, you need to manually turn it on by using a "DataUnlocker proxy" switch in the Property's settings. You can switch it back off anytime, and DataUnlocker's proxy will stop in 5-15 seconds.

If your health check is green and your web application is receiving some blocked traffic, you will start seeing proxied requests shortly on your property's Dashboard page. You can also visit your web application with ad blocker turned on to generate some blocked traffic.

For instance, if your web application uses Google Analytics and Google Tag Manager, and if you didn't turn off the "proxy all other services" setting, you should get the following services in the proxied services table:

• www.googletagmanager.com, the Google Tag Manager serving your configured tags.
• www.google-analytics.com, a Google Analytics domain for reporting analytics and downloading the analytics.js script (depending on how Google Analytics was installed).
• api.dataunlocker.com, an endpoint serving a tiny piece of data telling synchronous DataUnlocker script which services to proxy and which not.

Note that by default DataUnlocker will attempt to proxy all blocked requests, unless "proxy all other services" is off. Normally, proxying everything that fails is not what you want, as many failed requests will still be failed even after the proxy attempt. Hence, it is recommended to turn off "proxy all other services" once you have configured which services to proxy.

Proxied services have "Enabled" status and a green checkmark in the proxied services table. You can click on a table row to configure each particular service: to enable/disable a proxy for it, manage its settings and see last proxied errors, if there are any.

User management

DataUnlocker's Property can be shared between multiple accounts. You can choose an account to be a viewer or an owner. Viewers can only view the property but cannot change its settings. Viewers are also able to go through the DataUnlocker's installation steps (but they won't be able to change the property's routing option).

You can add as many users as you want, and you can add not yet registered users. Once they log in on DataUnlocker, they will be granted access to the property as configured.

Billing

How DataUnlocker measures proxied traffic

DataUnlocker charges you for the proxied traffic via DataUnlocker's proxy service. Only blocked traffic is proxied, hence you are only charged for requests which were unblocked.

DataUnlocker uses a pay-as-you-go prepaid billing model, meaning that you need to purchase the traffic first in order to use it for the proxy. Initially, you have 512 MB of traffic for testing and verifying that DataUnlocker will work flawlessly with your web application.

You are billed for each consumed byte of the DataUnlocker's proxy. DataUnlocker counts both incoming (upload) and outgoing (download) traffic to its proxy services against your prepaid traffic. To be exact, DataUnlocker sums up a reverse proxy request size (which can also include additional headers and hence be a few bytes more than the original request) with the DataUnlocker's proxy response size (the response your browser gets from DataUnlocker). The size of a request/response is a total number of bytes it takes, which includes both headers and its body.

You can view the current prepaid traffic spending trend on the Property Billing page. The remaining prepaid traffic is displayed either on a chart or in the Usage Summary table. When the prepaid traffic reaches 0, DataUnlocker proxy will stop working.

One-time traffic purchases

There are two payment options available: one-time purchase and automatic purchases. Both have the same price for prepaid traffic, which you can see on the corresponding cards.

DataUnlocker works with the secure, PCI DSS-compliant payment provider Fondy, which accepts various card payment methods like Visa, Mastercard, Google/Apple Pay, etc. Because DataUnlocker service currently reside Ukraine, you will be charged in UAH equivalent to the USD amount you are presented in the DataUnlocker Admin. The exact amount charged will be visible in your bank notification, which may also include your bank currency conversion fee.

After each payment, you will receive a confirmation to your email address, as well as you will be able to see the transaction in DataUnlocker Admin: both in your Settings or on the Property's Billing page. In case you abandon the payment page for any reason, you will also receive a follow-up email soon, or you will be able to open the payment page again from either Settings or Property's Billing page.

Please make sure DataUnlocker works for your needs before making a purchase. Purchase refunds are possible for any objective reasons but are also subjects of our terms and conditions.

Automatic traffic purchases

You can set up automatic charges from your card when Property's prepaid traffic is getting low. It is possible to define two types of automatic purchases:

• Purchase a fixed amount when prepaid traffic is below the specified value.
• Purchase a calculated amount of traffic when prepaid traffic is below the specified value.

When selecting one of these options, you will be provided with enough additional information about what, how and when an automatic purchase system will charge. For automatic purchases, you will need to add a new card to your DataUnlocker's account.

To avoid any strange charges, we made a very simple formula for computing automatic purchase amounts. It takes the amount of traffic proxied for the last 1, 3 or 7 days, depending on how long your Property's proxy is running, and assumes that the same amount of traffic per day will be proxied for the next X days, which you specify as the input. For example, if you have just set up your property, and it has already proxied 100 MB of traffic, the formula will pick "last 1 day" (because you have just registered your Property), and hence assume that you spend 100 MB per day. Therefore, if you specify "Purchase enough traffic to cover the next 30 days", the formula will calculate 3000 MB of traffic to purchase (100 MB x 30 = 3000 MB = 2.929688 GB).

Frequently asked questions

Does DataUnlocker support <INSERT YOUR PRODUCT NAME HERE>?

Yes.

DataUnlocker is a product-independent tool, which fixes the underlying common problem (content blocking), instead of "fixing" each blocked third-party product individually. However, there are a few considerations you should be aware of when using DataUnlocker, which effectively means that your third-party product is supported out-of-the-box but it might have a few inconsistencies when proxied. Note that only blocked (proxied) traffic is affected, which means you're always safe installing DataUnlocker.

What should I be aware of when using DataUnlocker's proxy?

DataUnlocker is here to take all possible technical and management overhead from you, providing a value of restoring your web application's functionality without any code modifications. While everything is expected to work out-of-the-box and you are always safe to install DataUnlocker regardless of the third-party products or analytics tools you're using, here is a definitive list of considerations you should be aware of when using DataUnlocker:

• DataUnlocker never influences your existing traffic; it works only with blocked traffic. This means you're always safe installing DataUnlocker. However, because DataUnlocker patches some browser APIs and it's still in beta, you would want to first try installing the script in your test environment.
• Some third-party services may not fully understand proxied requests and hence f.e. record an incorrect geolocation for proxied (previously blocked anyway) requests. For instance, for some third-party tools, you may start observing a great number of new users coming from f.e. Amsterdam, where some DataUnlocker's proxy servers are located. DataUnlocker tries to fix geolocation if there is a known way to do so (for instance, it fixes geolocation for Google Analytics). You can report such cases with your third-party analytics tools and we will add geo support for it, if it's possible. Note that DataUnlocker provides a "support grade" for each proxied third-party service, for your information:

• DataUnlocker's should work just fine for all known (and all unknown) analytics tools. However, it's <script> may not yet support some exceptional cases (for instance, tracked content in iframes or adding blocked elements/scripts via innerHTML). If you spot a blocked request which was not proxied (unblocked) by DataUnlocker, please, let us know, and we will support it very soon.
• Because proxied requests go from a limited set of (DataUnlocker's) IP addresses, proxing some third-party services may encounter throughput limitations. While this was not the issue so far, with some less known third-party services, unblocked requests may start getting for example 429 HTTP status codes in response. Please report this observations if it ever happens with your third-party services.
• Watch your email or subscribe to our announcements Telegram channel to get the latest information and receive updates regarding your Property.

I have privacy concerns. What data does DataUnlocker store about my property?

We do not store anything more than what you can see in the DataUnlocker Admin. Additionally,

• DataUnlocker does not store (nor even log) proxied requests content, with the exception of the last proxied request with 4xx or 5xx response codes, for the purpose of displaying it in the user interface and improving our service.
• DataUnlocker stores aggregated data about how many requests were proxied, their status code counts and traffic, for the purposes of displaying it in the user interface. DataUnlocker may aggregate this data across all properties to understand to which third-party services we can improve our service.
• We would never advertise the usage of DataUnlocker with your Property (website, web application) without the prior communication and an agreement from your side.
• See our privacy policy for more information.

After installing DataUnlocker, I still see ad blockers blocking my analytics. Does it even work?

Most likely, yes, in case you did the Property setup correctly, received the all-green health check result and enabled DataUnlocker in settings. You will still see ad blockers blocked requests count, as well as net::ERR_BLOCKED_BY_CLIENT errors in the developer tools, however, if you take a closer look, you will notice that each blocked request is now followed by an unblocked encoded request. Unblocked requests are usually not readable, because they are encoded.

However, to ensure that the DataUnlocker's proxy actually delivers an expected result to your data analytics tools (or any other third-party services), you need to check your analytics tool (or third-party service) yourself, because DataUnlocker doesn't store any data to tell what has happened and how exactly.

What if my URL prefix becomes blocked?

As also explained in how it works, DataUnlocker should be faster to reach you out before you will see a traffic drop in your analytics tools or third-party services. Changing a prefix URL is as easy as changing IP addresses of our servers, however, some manual work from your side may be required. There is also an opportunity to automate the URL prefix changes (for DNS-based routing method), while it's not yet generally available (let us know if you need it).

How to measure DataUnlocker's impact?

DataUnlocker is generally a proxy service which provides an informative Dashboard page to explore what was proxied and when. However, this may still not answer the question of, for example, how many users were really "unblocked".

Because no proxy can tell exactly what was the result of the request sent to the target server, DataUnlocker rather provides you with additional features which help you to measure the impact in your third-party services directly. Currently, DataUnlocker has implemented an additional feature for Google Analytics: read how to set it up below.

Add a custom dimension to Universal Analytics (old Google Analytics)

NOTE: DataUnlocker can inject a custom dimension to proxied hits. However, because some (not very used) ad blockers block only third-party scripts (but allow pixels/hits), you may see a slightly lower number of "unblocked" users in Google Analytics than the real number of unblocked users.

The guide below describes the process of setting up a custom dimension for Universal Analytics (Google Analytics prior 2021). Instructions for newer Google Analytics 4 are available below.

Setting up a custom dimension will allow you to split the traffic coming from DataUnlocker and from genuine sessions in your Google Analytics reports. First, you need to create a new custom dimension in Google Analytics:

You can give this dimension any name, for example, "DataUnlocker". Select the scope of this dimension to be hit, as DataUnlocker will inject this dimension on every hit.

After creating a dimension, you will see it in the table. Remember the dimension's index you have assigned to your new custom dimension. On the screenshot below, this index is 4.

Now, you just need to tell DataUnlocker which dimension to populate. On your property dashboard page, click on the www.google-analytics.com row and input the dimension name "dimensionX", where X is your dimension index (in this example, the index is 4, so the input would be "dimension4"). Note that if you have stats.g.doubleclick.net in the table of proxied services, you need to do the same by clicking on stats.g.doubleclick.net row.

Done! Now, using this custom dimension, you will be able to distinguish traffic in Google Analytics and measure, for example, the number of your website's visitors using ad blockers. The percentage of users using ad blockers can vary greatly depending on your audience, especially for the tech sector.

To do so, you can add a user segment which will include only users that were "unblocked". Start by navigating to any of the dashboards in Google Analytics (like Audience Overview), and click on "Add Segment", and then clicking on a "New Segment" button:

Click "Conditions" under "Advanced" header, then click on the condition dropdown and select your newly created dimension under "Custom Dimensions":

Ensure that the dimension's value you entered in DataUnlocker ("true" in this example) matches with the value you enter in the segment creation form. Then, type a name of the new segment and click "Save".

Now, you will be able to see the unblocked segment of your users. The chart below demonstrates that 17% of the 100% of users (the default segment "All Users") were using ad blockers. The actual number of ad blocker users can be slightly higher, as not all ad blockers block pixels/hits, while the Google Analytics dimension is added on a "hit" level when the "hit" is proxied.

Add a custom dimension to Google Analytics 4 (new Google Analytics)

NOTE: DataUnlocker can inject a custom dimension to proxied hits. However, because some (not very used) ad blockers block only third-party scripts (but allow pixels/hits), you may see a slightly lower number of "unblocked" users in Google Analytics than the real number of unblocked users.

Adding a custom dimension in Google Analytics 4 is very similar to adding a custom dimension to Universal Analytics properties, just with 2 differences:

• Adding a new custom dimension is located in "Events" left navigation menu
• You can use any dimension name and specify any event parameter, not necessarily in the form "dimensionX". It can be just "dataunlocker", for example.

NOTE: if you haven't fully migrated to Google Analytics 4 yet, and is using Universal Analytics along with Google Analytics 4, you can keep the dimension name (event parameter name) in the format "dimensionX", as described in the guide above. Thus, the dimension will be compatible with both Universal Analytics and Google Analytics 4. Below, "dataunlocker" is used as a dimension name (event parameter name).

To create a new dimension in Google Analytics 4, navigate to the Events page, which can be accessed via the Events left navigation menu. Click on the "Create custom dimensions" button.

Input any name, which will be visible in the Google Analytics dashboards. For the Scope, select "Event". In the event parameter input, use the name of the dimension you specified in DataUnlocker.

In the example above, "dataunlocker" is used as a value of the event parameter name. Ensure that it is the same value you have specified in DataUnlocker, in the "custom dimension name":

Now, by analogy to how it works for universal analytics, add a segment matching your custom dimension to see distinguish unblocked users in your reports:

Select your custom dimension by name from the dropdown menu, and put "true" (or whatever you specified in DataUnlocker) as a value of "Dimension values".

After some analytics is collected, you should see the comparison view of 2 segments: all users and unblocked users. Below is an example of how it will look like.

I have other questions. Who should I get in touch with?

Reach out to support@dataunlocker.com, or check this page for more information.