About Us Services Web Hosting Web Design Specials Current Clients
Contact Information
Brett Mitchell
(612) 964-4443
brett@ digeraticonsulting . net
awstats easy installation on TextDrive Since Urchin was bought by google and they decided not to let TextDrive use it at their new data center many TextDrive customers have been very upset about the lack of a good stats package. I have heard that mint is incredible, but I didn't want to pay the $30 while I am waiting for the TextDrive designed solution to be available. I tried using slimstat, but found that it did not have all of the features that I was looking for (no log analysis, only a php script manually inserted into html files). I find the combination of the real-time, daily-viewable, data provided by slimstat and the power of awstats to be invaluable. I will put a tutorial for slimstat online soon, but for now here are the awstat instructions. If you want the tutorial for slimstat use the contact form at the bottom of this page and bug me about it so I get to it sooner.
If you need a real-time stats solution or want to view a single day's stats, but need the log analysis of awstats, you can have both! Just include the slimstat php script in each of the pages you want real-time stats for and install awstats for your powerful statistical analysis. Since awstats analyzes your server logs, you don't have to add anything to your html files. Slimstat also can't provide statistics on non-html content like images, pdf's, flash, css, etc, but awstats can. This becomes even more important if you want data like total bandwidth or a list of your files with the highest bandwidth usage.
This guide will seek to provide enough information for beginner users, but simplify the commands for more advanced users. A tips section will be below each step for more information if you need it. You need to know how to use sftp, ssh, and a text editor to follow this guide. If you do not know how to use these tools, we do offer technology consulting services to help you.
Step 1: Download awstats Download awstats 6.5 from Awstats homepage on sourceforge or by clicking on awstats-6.5.tar.gz
* The current version (as of 4/7/2006) is 6.5. These instructions apply specifically to that version; however, I have no reason to believe any future versions will be significantly different. You should scroll down the the download section and download the file called "awstats-6.5.tar.gz". At the time of writing this article version 6.6 beta was available, but the ChangeLog didn't seem to show any new features worth running a beta release (at least for my purposes).
* MAC OS X USERS: If you are using safari on Mac OS X 10.4, like I am, you will probably find that your OS will automatically decompress the gunzip file one level and you will have "awstats-6.5.tar" on your desktop. This won't cause any problems, but you will use the filename "awstats-6.5.tar" in the next step.
Step 2: Upload and Extract the files Using SFTP, upload the "awstats-6.5.tar.gz" to your TextDrive account in: domains/[mydomain.com]/tmp/
* This tutorial will assume you are setting up awstats for '[mydomain.com]' on a TextDrive account for '[username]', so you will need to substitute your domain and user name where applicable. The directory tmp does not exist, so you will need to create it. For reference, the command $ mkdir [directory name] is used to make a directory.
If you have made it this far, you should have the file "awstats-6.5.tar.gz" in the directory: "/users/homes/[username]/domains/[mydomain.com]/tmp/".
* Each ssh command in this tutorial will start with a '$' to signify it needs to be typed in at the ssh command line, do not type the '$'.
ssh into your account and extract the files by typing:
$ cd domains/[mydomain.com]/tmp
$ tar -zxf awstats-6.5.tar.gz
Step 3: Put everything in the right place
We need to put the executables in the cgi-bin, setup the web access to your stats, and make a place for awstats to store its data.
Do this by typing:
$ cd awstats-6.5
$ mv wwwroot/cgi-bin ~/domains/[mydomain.com]/cgi-bin/awstats
$ mv wwwroot ~/domains/[mydomain.com]/public_html/awstats
$ mkdir -p ~/domains/[mydomain.com]/homes/awstats-data
* You can put the awstats-data anywhere you want, but the homes folder is a good secure place for it.
Step 4: Setup the awstats configuration file
Awstats comes with a template file called awstats.model.conf. We will modify a copy of this file to setup your domain. Do this by typing:
$ cd ~/domains/[mydomain.com]/cgi-bin/awstats
$ cp awstats.model.conf awstats.[mydomain.com].conf
Now we need to make some changes to this configuration file so it will work with the domain. You need to open this file in a text editor. I use "Mac SFTP Carbon" on the mac and "BBEdit" combined for my SFTP and text editing needs. BBEdit lets you edit a file on your server and then save it directly to your server very easily. You can also use a command line text editor like pico. There are some good tutorials on pico that you can find using google. I might write one later, but I don't use it much so we'll see.
Make the following changes to the configuration file by locating each line and changing it's value (remember we just put it in ~/domains/[mydomain.com]/cgi-bin/awstats):
LogFile="/users/home/[username]/domains/[mydomain.com]/logs/access_log"
SiteDomain="[mydomain.com]"
DirData="/home/[username]/domains/[mydomain.com]/homes/awstats-data"
DirCgi="/cgi-bin/awstats"
DirIcons="/awstats/icon"
StyleSheet="/awstats/css/awstats_default.css"
We now need to test the configuration file and import some data. Type the following:
$ ./awstats.pl -update -config=[mydomain.com]
If you get a bunch of lines about parsing records, you did everything right. If not, go and check where you put everything and where you told awstats it was located in the configuration file. All errors should be because of that. If you followed the instructions all permissions should be correct.
Step 5: Setup cron job
If you want awstats to automatically continue to analyze your logs, you need to setup a cron job. As I mentioned earlier, awstats does not provide real-time information, instead it analyzes your logs and provides a web interface to view its interpretation. Do not run this cron job more than once per day (especially if you have a lot of traffic) and don't setup cron jobs to update multiple domains simultaneously. Since log files are rotated daily, you should scan your logs a little before midnight each day.
Setting up a cron job is quite easy and can be done using the webmin interface.
First go to http://webmin.[yourserver].textdrive.com:80 and login. Then click on the 'system' tab and then select 'scheduled cron jobs'. Now click on 'Create a new scheduled cron job'
Enter the following:
Command: /users/home/[username]/domains/[mydomain.com]/cgi-bin/awstats/awstats.pl -update -config=[mydomain.com]
When to execute: Select the radio button 'Times and dates selected below'
Minutes: check the 'selected' radio button and select a number in range [50-59]
Hours: check the 'selected' radio button and select 23
Days, Months, Weekdays: leave the 'all' radio button selected
Click on 'save' in the bottom right corner
I hope this tutorial was helpful. You should be all set with awstats now. To view awstats in your web browser, go to: http://[mydomain.com]/cgi-bin/awstats/awstats.pl. This is an incredibly powerful program and there are many ways to use the data provided to make your website more efficient, user friendly, and profitable. Digerati Consulting specializes in helping businesses and individuals with designing and optimizing their websites.
Additional Options One more thing you could do is use a simple shell script to automatically decompress each apache log file in your logs folder recursively and then load the data into awstats. I will post a script and instructions for using it in the next few days.
You can also change many of the options in the awstats .config file like: removing your IP address from the logs so you don't affect your data, adding domain aliases so you can accurately see referrals for your alias domains, and adding an html link to your awstats page that instantly updates the data from the current log (real-time stats). I'll see what I can do for these instructions too.
Getting old data into awstats So, now that you have installed awstats successfully, you probably want to import all of your old logs so you can see your historic statistics. To do this, all we have to do is run a little shell script to systematically go through each old log, decompress it, and import it into awstats. The first thing to do is to get rid of your current awstats data because awstats needs data to be imported chronologicallyand we already imported some data. To do this, just go to /users/home/[username]/domains/[mydomain.com]/homes/awstats-data/ and delete everything. Don't worry, all this data is still in your log files, so it will come back when the script imports these logs.
* Note: before running this script, make sure your .config file is setup to your specifications. For example, if you wanted to remove your IP address from the awstats data or not display statistics for certain pages, you would want to do this before importing the logs because you won't have another chance to run it. If you decide that you made a mistake, you can always delete all the awstats data again and start over with the script. Unfortunately, this does require a significant amount of time and processing power, so try to get it right the first time. You can also go to the awstats page while you are importing data to check the status. If your website gets some traffic and it has been up for a while, expect a good half hour or more of time for this script.
WARNING: if you setup awstats to be updated from the browser, DO NOT click on 'update now' while this script is running. Additionally, make sure you don't have a cron job scheduled to run at the same time either.
You can paste this script into a ssh window or paste it into a text file with a .sh extension and run it.for log in $(ls -rt ~/logs/access*.gz)
do
echo $log
echo “zcat”
zcat $log > ~/tmp/access_log
sleep 20
echo “awstats.pl”
./awstats.pl -update -config=[mydomain.com] -LogFile=/users/home/[username]/tmp/access_log
sleep 20
done
I admit that I need to work on the formatting of this page a little, but I have been a little sleep deprived lately and haven't had the chance. Above is the much requested script to parse old log files. Let me know if you have any problems using it via the script below. I normally test everything a lot before posting it, but it is 5:30am and I need some sleep. If you try this let me know via the contact form below how it worked. More to come soon, feel free to bug me about putting more online and I'll get to it sooner.
Feedback
Contributions to the community
- Guide to setting up awstats at textdrive: awstats on TextDrive
- Custom implementation of Speed Test: Speed Test
