Scrawlr is short for SQL Injector and Crawler, a tool developed by the HP Web Security Research Group in coordination with the Microsoft Security Response Center in response to the widespread SQL injection attacks on the web.
“Scrawlr will crawl a website while simultaneously analyzing the parameters of each individual web page for SQL Injection vulnerabilities. Scrawlr is lightning fast and uses our intelligent engine technology to dynamically craft SQL Injection attacks on the fly. It can even provide proof positive results by displaying the type of backend database in use and a list of available table names. There is no denying you have SQL Injection when I can show you table names!”
Key Features of Scrawlr include:
Identify Verbose SQL Injection vulnerabilities in URL parameters
Can be configured to use a Proxy to access the web site
Will identify the type of SQL server in use
Will extract table names (verbose only) to guarantee no false positives
Scrawlr which is a free tool has a few limitations which are it’s crawl only upto 1500 pages, doesn’t support Blind SQL injection and will not test for Post parameters for SQL injection. Overall even with these limitations, it’s still a useful tool to check your sites to see if you’re safe from SQL injections.
Here’s a problem we had come across today. Whenever we tried inserting data into a certain table, MySQL kept throwing up the following error:
mySQL error: 1062
Duplicate entry '0' for key 1
After scratching our heads over this one, we tried to alter the table to set the autoincrement field to the next number and even that failed to fix the problem. Then hunting around the web, we found the cause of this problem. It seems that Mysql throws this error because the field type of the auto increment field is not large enough to hold the next value, so it tries to wraps the count back to ‘0′ where the is already a record with that value.
We changed the field from int to unsigned bigint which fixed the problem for us. So if you face the same problem, check if your autoincrement field has maxed the datatype for that field.
Bookmark and share:These icons link to social bookmarking sites where readers can share and discover new web pages.
How do you sort a varchar field which has the dates stored in it? If you try the usual order by on the varchar fields you’ll have trouble with the sort order, since MySQL will order this field like a varchar and you will surely have problems with dates being sorted this way. Take the following example of a table which has a field called feed_date which defined as varchar. Now ordering the results by feed_date will not give the correct order like in the following example:
Hmmm… 2006 coming in between 2008 doesn’t seem right does it? The only way to get MySQL to sort this result properly will be if we can get MySQL to understand this data as a date field. To to this, we can use the mysql function STR_TO_DATE. This function allows us to convert a string to date and allow us to specify in which format the date is present.
So to convert the current date format in the feed_date column which is in the mm-dd-yyyy format, we’ll have to use this syntax: STR_TO_DATE(feed_date, ‘%m-%d-%Y’). Now we can use this converted data to sort on:
mysql>SELECT id, feed_name, feed_date ,
STR_TO_DATE( feed_date, '%m-%d-%Y') AS date_for_sort
FROM `datetest` ORDERBY date_for_sort;
+----+-------------------+-------------+--------------+| id | feed_name | feed_date | date_for_sort|+----+--------------------+------------+--------------+|2| vinuthomas.com |12-23-2006|2006-12-23||1| VTs Tech Blog |1-23-2008|2008-01-23||3| sitesandsounds.in|3-12-2008|2008-03-12|+----+--------------------+------------+--------------+3 rows inset(0.00 sec)
Disclaimer: I know that doing this is going a round about way to getting this done, this is just a tip to help out when we have a problem like this and can’t really change the database field declaration and you have tons of data in the field which doesn’t confirm to the MySQL date format in a varchar field.
Stumbled across this interesting article at Leo Büttiker’s blog which tells why pagination on a web app is a database killer:
On the view of your database the worst thing you can do in your web app is paging. Paging is horrible in the view of performance. To explain let me take a little example:
SELECT SQL_CALC_FOUND_ROWS gb.*,
u.username,
u.uid,
u.geschlecht,
u.mitfoto,
[... some more fields...]
FROM member_gold_guestbook gb
LEFT JOIN users u ON u.uid=gb.uid_from
[... some more left joins...]
WHERE gb.uid_to=’22152′
AND visible=’1′
LIMIT 0,10;
That’s not that bad at all, but when you go to page 300 your database server will hat you for this. The database server has not only to calculate the 10 items you want to show but also all 3000 previous items.
Sure you may argue nobody will ever go to page 300. Somebody will not, but “googlebot” and his evil brothers will. And the bad thing is that you can, as long as you need paging, nothing do against it. There are just a few tricks that may reduce your server load a bit.
Last week I’d finally decided to move off from Windows on my work laptop. After the default installation off the 64 bit CD, here were some of the problems I found:
The default Wireless drivers didn’t seem to work at all.
The 64 bit version of the OS seemed to lack some of the softwares like Flash player and w32codecs.
The external monitor/projecter didn’t work off the default installation.
My development stack of LAMP had to be installed manually.
Very basic multimedia support.
I reinstalled the laptop with the 32-bit OS instead, to get a hold of the missing software on the 64 bit version. The wireless was still down. A quick google search landed me at the following blog, which had a great step-by-step instruction on how to get ndiswrapper and the windows version of the Broadcom wireless driver to work on the laptop.
Instruction for installing the Broadcom drivers through ndiswrapper: http://vale.homelinux.net/wordpress/?p=144
To get the external display working, I followed the instructions over at Ubuntu’s community help site: https://help.ubuntu.com/community/BinaryDriverHowto/ATI.
The default open source drivers didn’t have support for TV-out. The instructions I followed were quite simple:
Install linux-restricted-modules and restricted-manager provied in the restricted repositories: sudo apt-get update
sudo apt-get install linux-restricted-modules-generic restricted-manager Open the restricted drivers manager included in 7.04 “System -> Administration -> Restricted Drivers Manager” and select “ATI accelerated graphics driver”. This will hopefully enable fglrx in a painless way. If not, follow the instructions for Edgy.
Apart from the instructions above you’ll also have to install fglrx-control. At the shell prompt type in:
sudo apt-get install fglrx-control
After this you can access the ATI control panel by issuing the following command at the prompt.
sudo fireglcontrol
These took care of my basic necessities. Had my network and basic software installed. Being a LAMP developer, I had to get LAMP installed on the laptop. Here’s how to get it done:
The first two gets apache2, and the php modules installed. The next two get the php command line, php dev libraries and pear installed on the system. Next let’s get MySQL up and running:
apt-get install mysql-server mysql-client
This installs MySQL server and the client setup. To setup the mysql libraries for PHP5, run the following:
apt-get install php5-mysql
Linux still lacks an IDE for LAMP development, so while I search the net for a suitable candidate, I guess gedit or Quanta will work fine for coding. One of the candidates in the open source area for this task is PDT, I’m yet to evaluate it.
To get all the other multimedia goodies installed on my system, I used BUMPS.
If your company has some nagging legacy web applications which refuse to work on anything other than IE, you can also install IE on Ubuntu using ies4linux. Here’s a link which shows you how do go about this:
Be warned that having IE on your system legally requires that you have a valid license for Windows.
If you’re pulling out your hair wondering why the hell will some one install IE on Linux, it’s just that some nasty web-application developers make cross-browser compatible apps that work on any browser as long as it is IE!
Amazon’s S3 service seems to be getting into everything ‘online’ these days. Here’s an announcement by Mark Atwood of a S3 storage engine for MySQL 5.
It allows one to view and manipulate Amazon’s S3 storage service as
tables and items by MySQL. You can keep your blobs or large varchars
or truely huge datasets in S3, and then join the tables against your
local ones.
Here’s a quick tip in MySQL. Let’s say you have a table called browsercount like the one below:
id
browser
count
1
Internet Explorer
1
2
Opera
2
Let’s say you have a web-analytic application which has to add a browser into the table if it doesn’t exist, but update the count for the browser if the entry already exists. How do you go about that?
The usual way would be to first query the table to see if the entry exists, then fire an insert statement or an update statement based on the results.
Here’s how to perform the same action in one SQL statement:
INSERT INTO browsercount (browser,count) VALUES ('Firefox',1) ON DUPLICATE KEY UPDATE count=count+1;
This will work if the id field in the table above is declared as UNIQUE or is a Primary Key.
The query basically tells mysql to insert the data into the database, but if a duplicate key is found, increment the count field by 1. It’s much faster than firing up 2 sql queries from your PHP script
Bookmark and share:These icons link to social bookmarking sites where readers can share and discover new web pages.
This video is from one of Google’s Techtalk sessions by Jay Pipes
ABSTRACT Learn where to best focus your attention when tuning the performance of your applications and database servers, and how to effectively find the “low hanging fruit” on the tree of bottlenecks. It’s not rocket science, but with a bit of acquired skill and experience, and of course good habits, you too can do this magic!
Bookmark and share:These icons link to social bookmarking sites where readers can share and discover new web pages.
I’ve had to to look at this situation quite a few times on live servers, where the number of MySQL connections seemed to exceed the max limit of the server and PHP used to throw “Too many connections” error when a MySQL query was run. When we looked at the MySQL process list, it showed quite a few MySQL connections in the Sleep state. Here’s an article from MySQL Performance Blog on debugging such connections.
I see if frequently with web applications and it is often indication of trouble. Not only it means you may run out of MySQL connections quicker than you expected but it also frequently indicates serious problems in the application. If you do not use persistent connections and you have connection in Sleep stage for 600 seconds what could it be ? It may mean some of your pages take that long to generate (or might be the code simply gets into the tight loop and page never gets generated) it also could mean some of external Web Services are slow or not available and you’re not dealing with timeouts properly. Or may be you have several connections to MySQL server and right now running query which takes that long ? In any case it is something frequently worth looking at.
Here’s a cool utility for MySQL administrators which allows you to track the performance of your servers and also gives you performance enhancement tips. You can see a sample report from this tool over at: http://gert.sos.be/demo/mysqlar/
The MySQL Activity Report package is a tool to help MySQL database administrators to collect several database parameters and variables. These collected values can be used for server monitoring or performance tuning purposes.
One of the candidates in the open source area for this task is 
