linux – 嗅探移动流量数据

我现在正在为我们的移动部门准备一个自动化测试实验室,主要由 android设备组成.

对于第一阶段,我需要HTTP设备嗅探,我不能在设备上进行,因为它们并非都是root用户,并且并非所有设备都是4.0及以上,因此他们可以使用基于VPN的解决方案.

我正在考虑将在设备wifi“高级设置”中定义的代理,并且应该使用请求响应标头正文创建.txt文件(因为它没有特殊格式,纯文本应该对我来说很好).

HTTPS我会在以后需要时解决.

我的问题是哪个代理服务器最快设置和编写此类操作的脚本？
我想不会有超过10-15个设备可以同时发出请求,加载在我的设置中不会出现问题,但要确保每个设备都有自己的数据打印到文件中.

我知道鱿鱼存在和tinyproxy我知道apache有一个proxy_mod可能会做的伎俩,不知道它是否符合我的需要.
服务器机器是linux所以没有小提琴手来帮我.

解决方法

>使用wireshark的linux HTTP代理
>网络捕获

听起来你更喜欢使用linux HTTP Proxy方法,但我也包括网络捕获信息.

HTTP代理捕获

使用python-proxy,我在页面底部添加了源代码修改.默认情况下,python-proxy仅侦听localhost.脚本中的一个小改动使它可以监听所有端口：

if __name__ == '__main__':
    start_server(host='0.0.0.0')

在linux中保存文件后,您可以使用python PythonProxy.py作为unpriv用户执行现在将您的Android客户端配置为通过tcp / 8080上的linux服务器的IP进行代理.

python-proxy不支持HTTPS.还有像mitmproxy这样的其他python代理项目支持https,但是我没有那么多成功(然后我再次将所有内容部署在virtualenv中,这对于一些python项目来说效果不佳).

在Linux上代理SSL流量的另一个选择是CharlesProxy,但我从未使用它.虽然它是商业产品……

网络捕获

许多管理型交换机(即Cisco / Juniper / HP / etc)将一个以太网端口的流量镜像到另一个以太网端口,并且交换机配置非常简单.

|               |
           | <--  wifi --> |
+-------+  |               |  +---------+ Eth     +-----------------+
| phone |--+               +--| Wifi AP |---------| Ethernet Switch |
+-------+                     +---------+         +-----------------+

                                                ^^^^^^ Sniff here

此方法不支持解密https捕获.

### PythonProxy.py

# <PythonProxy.py>
#
#Copyright (c) <2009> <Fábio Domingues - fnds3000 in gmail.com>
#
#Permission is hereby granted,free of charge,to any person
#obtaining a copy of this software and associated documentation
#files (the "Software"),to deal in the Software without
#restriction,including without limitation the rights to use,#copy,modify,merge,publish,distribute,sublicense,and/or sell
#copies of the Software,and to permit persons to whom the
#Software is furnished to do so,subject to the following
#conditions:
#
#The above copyright notice and this permission notice shall be
#included in all copies or substantial portions of the Software.
#
#THE SOFTWARE IS PROVIDED "AS IS",WITHOUT WARRANTY OF ANY KIND,#EXPRESS OR IMPLIED,INCLUDING BUT NOT LIMITED TO THE WARRANTIES
#OF MERCHANTABILITY,FITNESS FOR A PARTICULAR PURPOSE AND
#NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
#HOLDERS BE LIABLE FOR ANY CLAIM,DAMAGES OR OTHER LIABILITY,#WHETHER IN AN ACTION OF CONTRACT,TORT OR OTHERWISE,ARISING
#FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
#OTHER DEALINGS IN THE SOFTWARE.

"""\
Copyright (c) <2009> <Fábio Domingues - fnds3000 in gmail.com> <MIT Licence>

                  **************************************
                 *** Python Proxy - A Fast HTTP proxy ***
                  **************************************

Neste momento este proxy é um Elie Proxy.

Suporta os métodos HTTP:
 - OPTIONS;
 - GET;
 - HEAD;
 - POST;
 - PUT;
 - DELETE;
 - TRACE;
 - CONENCT.

Suporta:
 - Conex?es dos cliente em IPv4 ou IPv6;
 - Conex?es ao alvo em IPv4 e IPv6;
 - Conex?es todo o tipo de transmiss?o de dados TCP (CONNECT tunneling),p.e. liga??es SSL,como é o caso do HTTPS.

A fazer:
 - Verificar se o input vindo do cliente está correcto;
   - Enviar os devidos HTTP erros se n?o,ou simplesmente quebrar a liga??o;
 - Criar um gestor de erros;
 - Criar ficheiro log de erros;
 - Colocar excep??es nos sítios onde é previsível a ocorrência de erros,p.e.sockets e ficheiros;
 - Rever tudo e melhorar a estrutura do programar e colocar nomes adequados nas
     variáveis e métodos;
 - Comentar o programa decentemente;
 - Doc Strings.

Funcionalidades futuras:
 - Adiconar a funcionalidade de proxy anónimo e transparente;
 - Suportar FTP?.


(!) Aten??o o que se segue só tem efeito em conex?es n?o CONNECT,para estas o
 proxy é sempre Elite.

Qual a diferen?a entre um proxy Elite,Anónimo e Transparente?
 - Um proxy elite é totalmente anónimo,o servidor que o recebe n?o consegue ter
     conhecimento da existência do proxy e n?o recebe o endere?o IP do cliente;
 - Quando é usado um proxy anónimo o servidor sabe que o cliente está a usar um
     proxy mas n?o sabe o endere?o IP do cliente;
     é enviado o cabe?alho HTTP "Proxy-agent".
 - Um proxy transparente fornece ao servidor o IP do cliente e um informa??o que
     se está a usar um proxy.
     S?o enviados os cabe?alhos HTTP "Proxy-agent" e "HTTP_X_FORWARDED_FOR".

"""

import socket,thread,select

__version__ = '0.1.0 Draft 1'
BUFLEN = 8192
VERSION = 'Python Proxy/'+__version__
HTTPVER = 'HTTP/1.1'

class ConnectionHandler:
    def __init__(self,connection,address,timeout):
        self.client = connection
        self.client_buffer = ''
        self.timeout = timeout
        self.method,self.path,self.protocol = self.get_base_header()
        if self.method=='CONNECT':
            self.method_CONNECT()
        elif self.method in ('OPTIONS','GET','HEAD','POST','PUT','DELETE','TRACE'):
            self.method_others()
        self.client.close()
        self.target.close()

    def get_base_header(self):
        while 1:
            self.client_buffer += self.client.recv(BUFLEN)
            end = self.client_buffer.find('\n')
            if end!=-1:
                break
        print '%s'%self.client_buffer[:end]#debug
        data = (self.client_buffer[:end+1]).split()
        self.client_buffer = self.client_buffer[end+1:]
        return data

    def method_CONNECT(self):
        self._connect_target(self.path)
        self.client.send(HTTPVER+' 200 Connection established\n'+
                         'Proxy-agent: %s\n\n'%VERSION)
        self.client_buffer = ''
        self._read_write()        

    def method_others(self):
        self.path = self.path[7:]
        i = self.path.find('/')
        host = self.path[:i]        
        path = self.path[i:]
        self._connect_target(host)
        self.target.send('%s %s %s\n'%(self.method,path,self.protocol)+
                         self.client_buffer)
        self.client_buffer = ''
        self._read_write()

    def _connect_target(self,host):
        i = host.find(':')
        if i!=-1:
            port = int(host[i+1:])
            host = host[:i]
        else:
            port = 80
        (soc_family,_,address) = socket.getaddrinfo(host,port)[0]
        self.target = socket.socket(soc_family)
        self.target.connect(address)

    def _read_write(self):
        time_out_max = self.timeout/3
        socs = [self.client,self.target]
        count = 0
        while 1:
            count += 1
            (recv,error) = select.select(socs,[],socs,3)
            if error:
                break
            if recv:
                for in_ in recv:
                    data = in_.recv(BUFLEN)
                    if in_ is self.client:
                        out = self.target
                    else:
                        out = self.client
                    if data:
                        out.send(data)
                        count = 0
            if count == time_out_max:
                break

def start_server(host='localhost',port=8080,IPv6=False,timeout=60,handler=ConnectionHandler):
    if IPv6==True:
        soc_type=socket.AF_INET6
    else:
        soc_type=socket.AF_INET
    soc = socket.socket(soc_type)
    soc.bind((host,port))
    print "Serving on %s:%d."%(host,port)#debug
    soc.listen(0)
    while 1:
        thread.start_new_thread(handler,soc.accept()+(timeout,))

if __name__ == '__main__':
    start_server(host='0.0.0.0')

（编辑：ASP站长网）