Update policy: “free for minor updates” vs “free for x years”

Some of you might not be aware that starting with version 2.0 back in 2009 we have decided to switch to “free updates for a year” policy. Information about this can be found on several places (order page, blog here and here, emails, etc), but it’s probably easy to miss. To make this transition as painless as possible, we didn’t enforce this policy until v2.6, and we have embedded this information inside the application starting with v2.5 (which is still free for all v2.x customers). Also, the policy in last several updates are actually unofficially stretched to two years. Still, it seems that pain cannot be avoided completely. 🙂

The reason for this decision was that we are very conservative with raising major version numbers, even though, combined, all improvements between few minor updates (for example between 2.0 and 2.2 or 2.3), might deserve to be declared as v3.0 (you can see the link to the change history at the bottom of the home page to see if you agree or not).

On the other hand, there might always be an unfair feeling on customer’s side if they bought v2.8 and then we decide to put a v3.0 label on the next update – they would not be able to use it. And if we declare it as v2.9, they would, so it might seem a bit unfair for customers and also hard for us to “draw a line”. We felt that, in this case, as what we call “minor updates” are actually “moderate updates”, giving them for free within at least one year is more clear and it works better for both sides. Each of you can decide for yourselves if the improvements are worth now, in few years or never.

Of course, we can’t make everyone happy no matter how hard we try. I understand that it might be frustrating when this period is over, especially if you were not aware of the policy upfront, so if any of you have concerns about it, I encourage you to drop an e-mail so we can find a mutually acceptable solution.

Web Log Storming 2.8: Spider Identification Update and Custom IP Resolving

Web Log Storming version 2.8 is available for download. Most important changes include an improvement in spider identification algorithm and a custom IP resolving. This update is free for everyone who bought after February 14th, 2011.

Spider identification

Even well established web crawlers don’t behave as they used to. We remember happy times when all decent bots (excluding shady ones) presented themselves in User Agent field, so it was easy to identify them while parsing log files.

Prior to 2.8, these weren't recognized as crawlersAs one of our users noticed and pointed out (thanks, John!), even big players like MSN Bot introduce themselves as regular browsers now. This wasn’t easy to notice without resolving IPs to domain names; even by close inspecting their sessions, you can rarely notice anything unusual (they look like regular visitors, with IE7 browser and different Windows versions). Only after resolving IPs, you can see their true nature.

That’s why, in this version, we have added a feature to define Spider Domains identification list, in addition to already existing Spider User Agents list. Domains list can contain IP addresses (wildcarded) or domain names. That way identification works regardless if user resolves domain names or not.

Accompanied with this change, we have also added a right-click menu option Add Selected to Spiders in Domains report.

Custom IP resolving

Custom IP resolvingSpeaking of IP resolving, we have added an additional tool as a nice addition to Manually edit host name * option (Professional version only). If you work in teams, now you can define URL to web script that will be used to get these custom names from shared web space, for example database.

This script should follow simple rules for getting and setting IP/domain pairs. You can see more details here.

* In case you are not already familiar with it, Manually edit host name allows you to assign any text to IP address and this text will appear in reports as it was a normal domain name. That way you can assign texts like “Our company”, “Prospective client #1”, “Hacker”, etc.

Download Web Log Storming

Web Log Storming 2.7

Another Web Log Storming update is ready.

+ Added search & replace for file names (Project Properties window)
+ Introduced access method column and parameter (GET, POST, HEAD…)
+ Goals are clickable now in Overview report
& Search Engine detection only in domain
& Drill-down from Directory report: subdirectories are not excluded anymore
* Firefox v10+ are now correctly recognized
* “https” in referred URLs is correctly processed now
* Problem with quoted search phrases
* First goal was not visible on Overview report
* Crash when copying Overview report to clipboard in some cases
* Few other minor problems

Links:

Web Log Storming website
Download an update

Web Log Storming 2.6 and discount opportunity

As mentioned before, in 2009 we switched to subscription based upgrade policy. By buying a license or subscription extension you get free updates for one year. After that, you can continue using the last version you own (forever or for time being) or extend subscription right away. Because we wanted you to get used to this idea, we prolonged an implementation for a while, but now it’s time: version 2.6 won’t work if you bought your license before Sep 27, 2010.

If you update to v2.6 accidentally and you don’t wish to extend subscription, please bear in mind that this wasn’t our intention at all and that we took all reasonable measures to avoid such misunderstandings (blog, newsletter and a warnings throughout the application). In that case, you can download latest “last free for all v2.x customers” version from here.

Most important tangible change is introduction of Bounce rate metrics; now you can see how much visitors leave after visiting just one page. This metrics is included in the Overview report, but also on pages, referrer and search keywords reports. This way you can easily see how well specific page or referrer is performing.

Another important change is eliminating problems with Windows 7 64-bit. The problem didn’t happen on all systems and it was related to a third-party software protection system that we were using in combination with some anti-virus/firewall software. Well, it shouldn’t happen anymore, and if it does, by all means, please let us know. If your old registration data isn’t automatically updated, please try to enter your key again.

We also updated Operating System and Browser lists and fixed some cosmetic and minor bugs.

Facebook page and a discount

Several days ago we finally created a Facebook fan page. It still doesn’t contain much information and we are now awaiting more “Likes”. So, if you already like Web Log Storming, why wouldn’t you make it official? 🙂 In return, you will get a 20% discount for any of future purchases related to Web Log Storming (which includes new licenses and subscription extensions). This offer probably won’t be here forever, but once you get your coupon you will be able to use it anytime, so make sure you don’t miss it.

Web Log Storming website
Download an update
Facebook page

How to track AdWords campaign without using cookies

People often think that it’s not possible to track successfulness of marketing campaigns without using cookies and/or JavaScript, which isn’t true. I will explain how it can be done with our Web Log Storming, but maybe the similar results could be accomplished with some other web log analyzer. Also, even though the example shows AdWords, you can track any other advertising results in a same way.

Setting up an AdWords campaign

The goal is to somehow differentiate visitors that came from different AdWords campaigns, groups and/or ads. It’s quite easy, actually, as only thing you need to do is to append some query text to Destination URL like this:

If you append “?anything” to URLs (including question mark, without quotes), this information will be included in server log files, while visitors will see the same content as if query is not there. For different campaigns and ad variations append slightly different query, for example:

  • https://www.mywebsite.com/?adw=test1
  • https://www.mywebsite.com/?adw=test2

Use it in Web Log Storming

After traffic starts coming in, Web Log Storming will be able to analyze this information and include it in reports. Easiest way to this is to enter “adw=*” into Parameters | File | Query field (note there is no question mark here). You can also use Lock button to keep this filter active as you switch reports.

Custom variables in Web Log Storming

While this parameter (filter) is active, any report that you select will be based on AdWords visitors only, making it easy for you to see how they perform.

Another tool that you might want to use is View | Based on IP only main menu option. If you select it, sessions from the same IP will be grouped as one, allowing you to examine it as a returning visitor.

Going further – using Goals

If you use Professional edition, you can also define Goals important for your business. Please read this article to learn how to do that.

With two Goals defined, this is what you’ll get if you choose Queries report:

Query Count Goal1 % Goal2 %
adw=test2 68 45 66.18% 8 11.76%
adw=test1 27 3 11.11% 27 100%

As you can see, for each defined AdWords destination URL (campaign, group or ad), you get metrics that show how they perform against specific goals. In this example, campaign Test1 works perfectly for Goal2, while Test2 works better for Goal1.

How accurate is this?

As you might know, identifying visitors by IP might not be perfect. There is no guarantee that visitor will have same IP next time he visits website, and there is no guarantee that he’s the only one using this IP (proxies). However, I must argue that cookies are not perfect for this job either. For example, visitors can allow session cookies only, automatically clear cookies after closing a browser or block cookies and/or JavaScript completely.

Anyway, we are all probably sure that less people use proxies these days than few years ago, more people are on a broadband with fixed IP addresses and more people intentionally block cookies and analytics scripts. In other words, try this and you might be pleasantly surprised.

Web Log Storming 2.5, some important information

As you might know, we have released Web Log Storming v2.5 a day ago. This post is a follow up that explains few important changes.

Update Subscription

Even though we’ve switched to optional yearly update policy starting with v2.0 (as it is and was indicated on the purchase page), we have decided to prolong activating it for a while. Main reason for waiting was to give you, especially to v1.x users, some additional time to get used to the idea.

Now that time has come. Version 2.5 now informs user about date when free updates period ends (see Help | About), but it will never cease to work. Next version, however, will cease to work for those who bought their licenses more than a year ago. You’ll have three options:

  • Continue using the last version that you are eligible to, forever, or until you eventually decide that improvements are beneficial enough for your needs
  • Buy an update subscription for another year (for about 40% of full license price)
  • Buy a lifetime update subscription (for full license price)

Of course, in case you decide to extend subscription months or years after the original one expired, your new free updates period starts from the date of the new purchase. On the other hand, if you decide to extend it before current one expires, new period will not start until the original expiration date.

Important: as version 2.5 is the last version that will be free forever for all v2.x users, we strongly suggest you to install it.

Spider detection

If you often use View | Human option, you might suddenly notice dramatically reduced number of visitors. Don’t worry, it doesn’t mean that your business is going downhill. 🙂 We’ve implemented smarter algorithm to detect which visitors / domains are behaving like spiders.

It would be easier for analysis if spiders and bots followed few simple rules: introduce themselves as such in User agent field, read and respect robots.txt file, etc. Although some do (like Google, Yahoo and similar legitimate spiders), some don’t care and/or are trying to full us in different ways (“referrer spam”, for example).

This more advanced algorithm tries to figure out if certain domains and visitors don’t behave like human beings. For example, if you get lots of hits on small number of different files from a single domain on regular basis over the time, it’s most probably a bot. Of course, exceptions are always possible (false negatives and false positives), but we think that we’ve managed to reduce mistakes significantly compared to previous version.

That said, if you notice some obvious false positive/negative, feel free to let us know. We will analyze these sessions and see what we can do to improve the detection further.

Links:

Web Log Storming v2.5

Web Log Storming v2.5 is available for download. This update contains several improvements and bug fixes:

  • Better spider detection (possibly reduced number of Human visitors in reports)
  • Spider visual indication (marked gray)
  • Problems with downloading log files from HTTPS
  • Problems with some User Access Control settings on Vista and Windows 7
  • In some cases referrer was not correctly recognized
  • Friendlier message when project file (.wls) doesn’t exist
  • Other minor and medium bug fixes and improvements

Links:

Web Log Storming home page
Download an update

Conversion tracking in Web Log Storming

Recently, a topic of conversion tracking came up in the professional forum. Specifically, one member asked how to include Google Analytics tracking code into third-party order pages to track conversions and original referring websites that bring customers to him. After that, another member complained that only about ~5% of purchases were registered by Google Analytics system, so he gave up. This means that ~95% of customers were completely ignored by stats, and you will agree that those couldn’t be spiders.

Now, 5% seems a bit too low and probably varies from case to case, but nevertheless, we shouldn’t be surprised. There are several reasons why this type of analytics isn’t accurate, and it only gets worse over the time. As global awareness rises, more and more users set up their browsers to reject third-party JavaScript and/or cookies (those that are not hosted on the website they’re visiting), and some even block them all. Moreover, there seems to be a lot of interest on how to deliberately block Google Analytics, by various browser add-ons or some other ways.

Goals in Web Log Storming

Setting up Goals to track conversion ratios in Web Log Storming is pretty much straightforward:

  1. Identify the page that represents your goal, type something like “/my-goal-page.html” into File parameters wildcard and hit Enter (or F5) to refresh report.
  2. Now click “Add this as goal” link at the bottom of File parameters pane, set additional options if you wish and click OK button.

That’s pretty much it. From now on, you will see conversion percentages on most reports available in Web Log Storming. You can watch this video or screenshots to see how this looks like.

Tracking when goal page is on another website

If you are partnered with third-party payment services to process your orders, you won’t be able to get raw log files to analyze them in Web Log Storming, but there’s a rather simple solution. Usually, partner services provide means to include some custom HTML into their standard order page.

  1. Create 1×1 pixel transparent .gif image and upload it to your web server (for example, /goals/my-goal.gif). You can right-click and save this one, if you wish.
  2. In partner’s custom HTML include this code:
<img src="https://<www.mywebserver.com>/goals/my-goal.gif" />

From this point on, set up Web Log Storming similarly as in previous example. Each time visitors hits the goal page on the partner’s website, this small image will be requested from your web server, which means that you will get this information in your raw log files, and the image itself will be practically invisible for users.

What if the other website is secure (https)?

If your partner’s order pages are located on the secure server (which they are, I hope 🙂 ), browsers could show a not-so-nice warning about mixing up secure and non-secure elements. In this case, you will need to host this transparent .gif on a secure server too. If you don’t already have one, you can create a SSL certificate yourself or buy a cheap one here (under $10/year). Note that for this purpose it doesn’t have to be a super-secure server. You won’t transfer any of sensible data to it, so a formal SSL certificate will suffice.

Obviously, in this case, the HTML code should be changed to:

<img src="https://<www.mysecurewebserver.com>/goals/my-goal.gif" />

As your main website server and secure one will probably write log hits to different files, depending on your original configuration, you might need to add another Log file location in Web Log Storming’s File properties window.

Links:

Web Log Storming home page
Download 30-day free trial

Web Log Storming v2.3 available for download

Web Log Storming v2.3 is now available for download. Changes in this release contain several new features that will make Web Log Storming easier to use, some improvements and additions and, of course, few bug fixes. As this is a free update for anyone who bought v2.x, we strongly recommend you to download and install an update.

Links:

Web Log Storming home page
Download an update