stas'den - perl

The Pirate Bay un-SSL

Thu, 31 Jul 2008 07:05:20 -0700

Theory

Recently, the world saw The Pirate Bay offering SSL encryption on their server. This means that your ISP won't know anymore which torrent you are downloading, right? Wrong.
HTTPS is quite useless for protecting static and public content. By static, I do mean the .torrent file itself. It is always the same. By public, I do mean than one doesn't need any kind of authentication to pick up the content. It's always the same, for everyone. For crawlers, too.
So, one could easily index (a portion of) The Pirate Bay torrent database by the Content-Length. Then, one could intercept some encrypted traffic between some machine(s) within his/her network and the torrents.thepiratebay.org server. Knowing both (encrypted) request and response lengths, it is possible to get a quite reliable list of matches from the previously indexed torrent list.

Practice

Don't try this at work, or you might hurt yourself

Use Wireshark to capture some torrent downloads. Torrents are hosted on a separate server, which makes the task easier yet. Just use the following capture filter: "tcp and port 443 and host torrents.thepiratebay.org"
Now, just go with the stream ("Follow TCP Stream" for the packet you suspect belongs to the torrent download. This will create another filter, just like "(ip.addr eq 192.168.0.10 and ip.addr eq 83.140.176.156) and (tcp.port eq 2157 and tcp.port eq 443)")
Just save the displayed stream anywhere else (pcap1.pcap sounds nice)
Now, use my quick&dirty TPB-TLSlen.pl Perl script to get the request/response lengths:
```
perl TPB-TLSlen.pl pcap1.pcap
```
Yeah, I know, it is nasty. It only supports the TLS cypher. And it simply calls the tshark (the command line version of Wireshark) to parse it's output.
Now, just paste the REQ and RES values below
(note that the REQ value is optional, setting it to 0 simply ignores the request size for matching)

Note that you are able to fine-tune the maximum and minimum header sizes. For the response, the headers are almost the same all the time. The only thing that varies is the decimal representation of the file length and age. (Un)fortuately, the request headers do vary for different browsers and referring pages. However, knowing the request size still helps a bit, specially if the torrent's filename was huge

Precision

The following size distribution chart was generated using the database with ~165K torrents:

The most common torrent size is ~14 KB, and it's easy to figure out that such torrents represent the shared 700 MB files
There's also a major peak for the 454 bytes torrents. However, bigger torrents are less common, thus, the size detection technique becomes more precise. Now, the average "distance" between torrent sizes is ~44 bytes (at least for the sample I've collected). So, adding a cookie with the random size up to 128 bytes will disrupt the size matching detection a lot. The request size disruption is even easier: the largest torrent URI I've found was 150 bytes-wide. Thus, padding every request URI to match 150 characters is enough to make the requests completely indistinguishable. Joining the pieces (the padding add-on strings are bold):

GET /4319199/[a4e]Ghost_in_the_Shell_TV_01-26.4319199.TPB.torrent?nVM2UGfcG533un4ym70eT2
9r0WwBLYdmFCNN+UTV/hiJ7EAXdFU5KfdWHpkB5lXaCmITsACKOPVyjmpbaOB+CrI5 HTTP/1.1
Host: torrents.thepiratebay.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208
 Firefox/3.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://thepiratebay.org/recent
Cookie: language=pt_BR; country=BR; PHPSESSID=ad6cb7e414c8dc88e0c2444f6215165a

HTTP/1.1 200 OK
Content-Type: application/x-bittorrent
Etag: "2198642509"
Last-Modified: Mon, 28 Jul 2008 22:28:59 GMT
Server: lighttpd
Content-Length: 91601
Date: Mon, 28 Jul 2008 22:37:56 GMT
X-Varnish: 108010229 107999438
Age: 253
Via: 1.1 varnish
Connection: keep-alive
Set-Cookie: p=68eOfxOC7JwBYcMe1RJWC4Z5PV/lJzqJORW8KROPMH9zQhszSjFnRp2tsNWEoyabWAloneUaoz
MxYtx4hoM9MZUKE/7wGzC3ZKLEZdppG4og3W; expires=Mon, 28-Jul-2008 22:37:56 GMT; path=/;
 domain=torrents.thepiratebay.org

(binary torrent data)

Solution

Use a constant padding in the .torrent files. This messes things a bit, but stills ineffective. The only advantage is not messing up with the server
Patch the lighttpd server so it sends a non-lasting cookie with a random size.

Thanks

MEGA Hospedagem, for the network resources provided for this tiny research
http://www.warchalking.com.br/, for the inspiration

Encrypted session data

Possible matches

The Pirate Bay URL	strlen(URI)	torrent size
0 matches
Torrents indexed: 1011548

"keyboard injection" under Linux

Wed, 15 Aug 2007 17:20:27 -0700

What to do if you want to pass some data into STDIN of some process, and it doesn't accepts it?! One of the most (un)famous example is ssh; who never tried to do a "echo password | ssh host"?! So, you just want to inject characters as there were someone typing them. Remember: TIOCSTI ioctl() call is your friend
Just google it to see what it does. And now, some actually working code:

#!/usr/bin/perl -w
use strict;
use Fcntl;
use constant TIOCSTI => 0x5412;

unless (@ARGV >= 1) {
 print "usage: $0 [vt] <input>\n";
 exit;
}

my $vt = shift @ARGV;
my $buf = join '', <>;
&writevt ($vt, $buf) || die "can't write to $vt: $!\n";
exit;


sub writevt {
 my ($vt, $buf) = @_;
 sysopen (VT, $vt, O_RDONLY) || return 0;
 for (my $i = 0; $i < length $buf; $i++) {
 ioctl (VT, TIOCSTI, substr ($buf, $i, 1)) || return 0;
 }
 close (VT) || return 0;
 return 1;
}

And this is how it works (Linux-specific!!!):

[stas@home stas]$ echo id | ./catvt.pl /dev/pts/0
id
[stas@home stas]$ id
uid=?????(stas) gid=?????(stas) groups=?????(stas)

Kinda awesome

Formulário CEP/CPF/CNPJ

Wed, 15 Aug 2007 16:31:52 -0700

(tente preencher os campos acima; a informação é atualizada instantaneamente)

Este projeto é o sucessor do CEP-2-City. É um formulário online que:

Verifica a validade do número CPF
Verifica a validade do número CNPJ
A partir do CEP, deduz o endereço completo (Cidade/Estado/Bairro/Rua)
A partir do CEP, deduz o código DDD da região

O banco de dados utilizado é compilado a partir de diversas fontes. Se não constar a informação da rua, então somente a cidade é retornada. A interface com o banco de dados foi implementada usando Perl e PHP, e pode ser acessada via CGI, Flash ou AJAX. O sistema de busca é extremamente eficiente, e não necessita de MySQL. O tamanho do banco de dados é cerca de 60 MB, e a performance chega a milhares de consultas por segundo.

Portanto, eis uma solução bastante símples, flexível e eficaz para o cadastro de clientes. Já utilizei este sistema numa pesquisa que conduzi, e asseguro que me poupou bastante tempo. Para realizar as consultas, basta acessar a seguinte URL (com a devida substituição do CEP): http://sysd.org/brloc/brloc.php?cep=05437000, e processar a string retornada com a função parse_str() (em PHP).

Se tiver interesse no banco de dados em si, entre em contato!

X11 FrontEnd for the Rio Utility

Mon, 08 Jan 2007 04:37:08 -0800

X11 FrontEnd for the Rio Utility v1.07 made by guys from the Snowblind Alliance. Surely not the best nor the most beautiful of all GUIs made to manage the famous Rio MP3 Player, but I tried to implement some of the ideas that the "real" managers doesn't implement. List of some "interesting" features:

2 windows side-by-side like in Norton Commander legacy managers
treats M3U playlists as directories
displays the space remaining on the device as you select files to upload
current file & overall progress bars
realtime display of the transfer speed

Note that you need Tk8.0 extension for Perl (and the Perl itself) to get this frontend running! The GUI works under ActivePerl Win32 environment out-of-shelf, but I haven't tested if the interaction with the rio.exe is OK. You can download it here and test for yourself! However, I would suggest you to use my Diamond Rio PMP300 FS-plugin for Total Commander.

P.S. - if you get problems downloading files with long names, apply the patch I provided below on the Rio Utility source (not XRio itself!) and recompile it.

CEP-2-City

Wed, 03 Jan 2007 12:55:33 -0800

Módulo Perl que obtém o nome do município (do Brasil) a partir de um CEP dado. Exemplo:

#!/usr/bin/perl
use CEP;

# inicializa
my $cep = new CEP;

# $city será referência para array com estado/cidade
my $city = $cep->city ('12.437-660'); # processa somente os dígitos numéricos (0-9)

if ($city) {
    # $str será uma string no formato Cidade/ESTADO
    my $str = CEP::city_string ($city);
    # filtra os acentos e imprime em caixa alta
    printf "esse CEP pertence a [%s]\n", uc CEP::normalize ($str);
} else {
    print "CEP não encontrado\n";
}

exit;

Conforme pode ser observado no exemplo dado acima, este é um módulo orientado a objeto. A instância do objeto CEP é criada com new CEP. Durante a inicialização, é construído um array com a lista ordenada das faixas de CEPs (isso pode demorar, portanto é uma boa idéia criar uma única instância e reaproveitá-la indefinidamente). O método que realiza a busca binária pelo CEP e retorna o nome do município é city(). O seu único parâmetro é o número CEP propriamente dito. Somente os dígitos numéricos serão considerados. city() retorna uma referência para array que contém o nome do estado e o nome da cidade, nessa ordem. Inclui também uma subrotina city_string() que recebe como parâmetro o array retornado pelo city() e retorna uma string no formato "Nome da cidade/ESTADO". E, finalmente, quando os acentos são desnecessários, pode-se filtrá-los com a função normalize(), que recebe uma string com acentos e a retorna sem acentos

P.S. - cuidado com os line-endings do arquivo CEP.pm! Se o segmento __DATA__ for salvo com CRLF, o módulo apresentará comportamento estranho em sistemas UN*X!
P.P.S. - Este projeto tem agora um sucessor (que consegue obter informação sobre Rua/Bairro e até mesmo DDD regional)! Confira!

ACFTools (for X-Plane)

Tue, 09 May 2006 21:09:12 -0700

The Plane-Maker from X-Plane v7.x is fun and easy to use... Although I found it a bit limited to edit my planes' fuselage. 20 cross-sections aren't enough, sometimes. It would be nice to export the fuselage shape into any 3D-editor and fine-tune it there... This is exactly what my ACFTools are supposed to do. Please note that X-Plane v8.x is unsupported, unless you create & edit a v7 model and then import it to Plane-Maker v8. But I believe this isn't a problem: X-Plane v8 is able to use OBJ format files (which can be imported/exported by AC3D and Blender plugins) as parts of an aircraft, thus, extremely weird shapes are perfectly possible without even touching the ACF file.
Now, what exactly ACFTools does? It is able to process the file that stores all the aircraft data (except the textures), in the binary ACF format, and dump all the data in the human-readable TXT format. It is the same format as for Tony Gondola's ACF2Text utility, which inspirited my ACFTools. Then, ACFTools can extract the 3D model of the aircraft from this data, and export it in the AC3D format. After that you can edit the aircraft's shape as you want using AC3D and make the inverse process: from .ac to .txt, and then from .txt to .acf. Here comes the brief set of ACFTools features:

Convert from ACF (aircraft) & WPN (weapon) formats to TXT format and vice-versa.
Extract AC3D model from TXT file, along with wings and propellers (using the real airfoils!).
Import AC3D model into TXT file (fuselage, nacelles, wheel fairings & external fuel tanks only)

3D model extractor is also useful if you want to use X-Plane generated aircrafts somewhere else; as in your own Computer Graphics compositions.
ACFTools was written in Perl, thus it is able to work under Linux, MacOS & Windows. And it is Open-Source & highly modular, so you can modify it as you want, or use my functions for your own needs. When you start ACFTools, this is what you get at your console:

##############################################################################
[ACFTools v0.62a] Set of tools to play with ACF files outside of Plane-Maker
Perl script and modules coded by Stanislaw Pusep 
Site of this and another X-Plane projects of mine: http://xplane.sysd.org/

Allows you to:
 * export X-Plane (www.x-plane.com) aircraft data files to human-editable
   plaintext format and 3D mesh editable in AC3D modeler (www.ac3d.org).
 * import plaintext/3D mesh back to ACF file.
##############################################################################

Usage: acftools.exe  [parameters]
 o Commands:
	-extract [DEF]	: extract TXT from ACF (opt: using DEF definition)
        -generate       : generate ACF from TXT
        -merge          : merge body from AC3D file to TXT
 o Parameters:
        -acffile FILE   : name of ACF file to process
        -txtfile FILE   : name of TXT file to process
        -ac3dfile FILE  : name of AC3D file to process
        -noorder        : DO NOT sort vertices while merging bodies
        -noac3d         : DO NOT generate AC3D
        -(min|max)body N: write all bodies in specified range to AC3D
        -force LIST     : force extraction of bodies LIST (comma-separated N)
        -normalize N    : normalize wings to N vert/surface (N>=2 or no wings!)
 o Notes:
        * You can use abbreviations of commands/parameters (-gen or even -g
          instead of -generate).
        * The only required parameter for "extract" command is -acffile.
          Both -txtfile and -ac3dfile are derivated from it.
        * "generate" command and -txtfile has the same relation.
        * By default "extract" uses the latest DEF file.
        * "generate" doesn't need DEF at all (it is implicit in TXT)
        * If file to be created already exists backup is made automatically.
 o Examples:
        acftools.exe --extract=ACF700 --acffile="F-22 Raptor.acf"
        (extract 'F-22 Raptor.txt' from 'F-22 Raptor.acf')

        acftools.exe -e -acf "F-22 Raptor.acf"
        (same as above)

        acftools.exe -me -ac3d ladar.ac -txt "F-22 Raptor.txt"
        (merge *single* 3D body from 'ladar.ac' to 'F-22 Raptor.txt')

        acftools.exe -g -txt "F-22 Raptor.txt"
        (reverse operation; generate 'F-22 Raptor.acf' from 'F-22 Raptor.txt')

The most important thing you should know is that to convert from the binary ACF format to TXT, ACFTools uses one of the files stored in the defs directory. By default, it is a ACF740.def, which works for X-Plane v7.40 and later. You can specify any definition file you want, as shown on the screen above. You can also write your own definition files, if you have the structure specification. Note that to convert TXT back to ACF, no definition is necessary, as the TXT format itself holds all the data necessary to build the binary file. The second thing is to specify the file that ACFTools should convert. The name of the converted file and every of the intermediary files will be generated from this name. When converting from ACF to AC3D formats, TXT file is generated automatically. Please note that when you want to put the edited aircraft part back into the .acf file, you must save this part as a .ac separate file, then merge it into the TXT file, and after that convert it into ACF! Yes, it would be nice to have a GUI to automate the entire process, but I'm too lazy to do that! Anyway, may the source be with you
I strongly recommend you to read the docs\README.txt file contained in the ACFTools package before you start using it!

I.E. 'hidden image' maker

Fri, 05 May 2006 13:47:18 -0700

If you are using Internet Explorer (any version able to load this page , just select the above image (or go to "Edit => Select All", or press Ctrl-A). You may see a watermark with white letters forming a word "TEST". This only happens in IE, due to the specific way it masks the selected images to show their selected state. Firefox, for example, simply tints the image with bluish color. There are lots of jokes one can make using this simple principle. For example, all images on your homepage may have some "subliminal" messages. Adobe PhotoShop is perfectly suitable to make such 'Hidden Images' by hand; in fact, my inspiration was a tutorial made by Matt Kment to do this. But if you are too lazy and/or do not have PhotoShop installed (UN*X users, for example), you can use this small program I wrote in Perl to automate the "steganographic" process. I wrote it using ImageMagick^TM library (so it's required to successfully run the program). It is a command line program, but it is simple enough to use, and it has an advantage to be integrable into your own scripts/programs (you can even setup your server to embed your company logo into all graphics of your site, but this will overload the server's CPU and RAM too quickly). I called my program "Internet Explorer Mask'O'Matic". Take a look at it's startup screen:

Internet Explorer Mask'O'Matic v1.0 by Stas 
Grab yours at http://sysdlabs.hypermart.net/ 

Inspirated by some mad stuff by Lem0nHead
Based on Photoshop 'Hidden Image' Guide (http://www.atomicwienerdog.com/ot/)
made by Matt Kment & suggested to me by xfalmp


ERROR: please give us --visible


 * Usage: MaskOMatic.pl 
 * Options (note that you can use syntaxes like --vis= --hid -out -p):
 --visible filename of image normally seen (required)
 --hidden filename of image seen when selected in IE (required)
 --output filename to write out, format is selected automatically
 using suffix provided (required)
 --percentage float value between 0 & 100; how much of hidden appears
 (optional, defaults to 50%)
 --quality JPEG/MIFF/PNG compression level (optional, DON'T USE!)
 --contrast flag, internal contrast reduction (optional)
 --test filename to dump preview of selected image (optional)

 * Notes:
 # Visible & hidden images doesn't need to have same size, when size
 doesn't matches then hidden image is rescaled using Lanczos filter
 # A huge set of image formats is supported (JPG, GIF, PNG for example)
 but I strongly advice you to write output in loseless format *only*
 (BMP, PNG, TGA) and *then* fine-tune contrast/brightness & save
 compressed in your favourite image editor (like GIMP)

 * Example:
 MaskOMatic.pl --vis bush.jpg --hid death.jpg --out sublim.bmp

Only 3 options are obligatory: --visible, --hidden & --output. You can abbreviate them as -v, -h & -o, respectively. "Visible" is an image that user sees by default. "Hidden" is what user sees when he/she selects the image. And "output" is the resulting image you can embed into your pages. It is also possible to use the "--test" option to preview the selected state of the output image without starting Internet Explorer. If it looks bad, you may play around with "--percentage" option, which specifies the 'weight' of the hidden image. The higher is this value, the greater is the hidden image visibility. If that's not enough to hide your image well (some things are pretty difficult to hide!), you can try to tune the output image with "--contrast" option, but I suggest you to use some GUI program to do that as you get the visual feedback instantly. I also suggest you to always save your output in the non-compressed lossless format (like BMP, TGA, TIFF), and then fine-tune the compression in some program specialized in it (GIMP has a nice compressor with visual feedback).

Alternatives:

Hidden Image Photoshop tutorial
Magic Image Generator C#/VS.NET2003 program (with source)

GibCounter QW stats

Wed, 19 Apr 2006 21:48:22 -0700

GibCounter is a tiny yet quite useful game statistics generator for QuakeWorld games. It operates parsing frag*.log files generated by the QuakeWorld game server. If your server doesn't generate such a log files by default, you may enable this feature executing the server as follows:

qw-server +set fraglogfile 1

Of course you can also edit your server's .cfg files to enable frag logging. You may run GibCounter on the same machine (and, on UN*X systems, as the same user) that runs the game server. If it's a QuakeForge server, then GibCounter will locate log files automatically at the path $HOME/.quakeforge/qw. For the different ports of the QuakeWorld server, or a QuakeForge server operating as a different user, you may specify the location of the qw directory manually. To do that, simply pass the new directory as an argument to GibCounter (using your favorite command line shell):

perl gibcounter.pl /home/qserver/.quake/qw

GibCounter outputs the generated HTML code directly to STDOUT. So, if you're going to add it into your server's crontab, don't forget to pipe the output to some file! For example, the following crontab line will regenerate GibCounter game stats every 30 min and make it available through the URL http://yourserver.com/~youruser/gibcounter.html (file paths & crontab format may be different on your system; so ask your system administrator if unsure):

0,30 * * * * perl $HOME/gibcounter.pl > $HOME/www/gibcounter.html

GibCounter is also highly theme-able: CSS style of almost every element may be changed within gibcounter.css file. Of course, you can also edit the HTML template which is contained inside the Perl source itself
The game statistics page generated by GibCounter is self-explanatory. On the top, it will show players ranking, sorted by (guess what?!) the frags they scored. Frags are computed as: kills minus suicides. GibCounter also computes how many times each player was killed by others. And, on the bottom of the generated page, some computed totals are shown. This includes the period of time for which the stats were made, the top fragger (the best) & the top fragged (the worst) players. Please note that players with default nicknames, such as "unnamed" or "user-#", are automatically excluded from being processed, simply to avoid useless information bloat (as many different players would be rated as one, with very high kill/death number)!
By the way, GibCounter preserves the colorization of the graphical font in players' nicknames (Quake console is able to print some ASCII characters in white, orange, gold & brown colors), and translates all symbols into readable ASCII.
So, after all, how does GibCounter-generated page looks like?! Look for yourself, here are some example stats. Also, feel free to modify the program itself to fit your own needs!

ptBR nums2words()

Wed, 19 Apr 2006 21:37:14 -0700

Uma função que programei para retornar strings com números escritos por extenso. Por exemplo: nums2words(2006) retorna "dois mil e seis".
Não que não exista mais nenhum algoritmo para isso; acontece que quis criar o meu próprio de tal forma que ele seja de fácil implementação em C, e que seja portável para outras linguagens, no sentido mais extenso (tanto linguagens computacionais, como Perl, Java, PHP; como linguagens humanas, como inglês, espanhol, russo).
Funciona da seguinte maneira: os números são "decompostos" em sub-unidades maiores possíveis às quais nós, humanos, damos nomes. No exemplo do 2006, há 2 unidades: "dois mil" e "seis". Essa operação é feita através do velho e bom resto da divisão, sem ter que converter o número em string e processá-la dígito por dígito.
Na realidade, comecei programando-o em Perl, depois traduzi para C, refinei, otimizei, aí traduzi de volta para o Perl
Não sei se está gramaticalmente correto; porém está perfeitamente compreensível. Acredito que o mesmo algoritmo poderá ser reaproveitado para inglês e russo. Porém isso fica a cargo do leitor!

	REQUEST	RESPONSE
SSL size:
Min header length:
Max header length: