博客园_Gavin

如何在Linux下实现50万并发

2011-04-04T13:16:00Z

网络上发现的一篇文章，虽然对问题介绍的不是特别深入，不过作为了解性的文章很不错。标题颇有标题党的感觉。。。

或参考

http://gavin1992.gotoip2.com/papperdetails_tech.php?pid=15&tit=如何在Linux下实现50万并发%20C500K

原文地址：

http://urbanairship.com/blog/2010/09/29/linux-kernel-tuning-for-c500k/

Linux Kernel Tuning for C500k

By Jared "Lucky" Kuolt • September 29th, 2010 • Posted in Android • 24 Comments

Note: Concurrency, as defined in this article, is the same as it is for The C10k problem: concurrent clients (or sockets).

At Urban Airship we recently published a blog post about scaling beyond 500,000 concurrent socket connections. Hitting these numbers was not a trivial exercise so we’re going to share what we’ve come across during our testing. This guide is specific to Linux and has some information related to Amazon EC2, but it is not EC2-centric. These principles should apply to just about any Linux platform.

For our usage, squeezing out as many possible socket connections per server is valuable. Instead of running 100 servers with 10,000 connections each, we’d rather run 2 servers with 500,000 connections apiece. To do this we made the socket servers pretty much just socket servers. Any communication between the client and server is passed through a queue and processed by a worker. Having less for the socket server to do means less code, cpu-usage, and ram-usage.

To get to these numbers we must consider the Linux kernel itself. A number of configurations needed tweaking. But first, an anecdote. （下面开始讲故事....）

The Kernel, OOM, LOWMEM, and You

We first tested our code on a local Linux box that had Ubuntu 64-bit with 6GB of RAM, connecting with several Ubuntu VMs per client using bridged network adapters so we could ramp up our connections. We’d fire up the server and run our clients locally to see just how many connections we could hit. We noticed that we could hit 512,000 with our Java server not even breaking a sweat.

The next step was to test on EC2. We first wanted to see what sort of numbers we could get on “Small” instances, which are 1.7GB 32-bit VMs. We also had to fire up a number of other EC2 instances to act as clients.

We watched the numbers go up and up without a hitch until, seemingly randomly, the Java server fell over. It didn’t print any exceptions or die gracefully—it was killed.

We tried the same process again to see if we could replicate the behavior. Killed again.

Grepping through syslog, we found this line:

Out of Memory: Killed process 2178 java

The OOM-killer killed the Java process. Having watched the free RAM closely, this was odd because we had at least 500MB free at the time of the kill.

The next time we ran it we watched the contents of /proc/meminfo. What we noticed was a steady decline of the field “LowFree”, the amount of LOWMEM that is available. LOWMEM is the kernel-addressable RAM space used for kernel data. Data like socket buffers.

As we increased the number of sockets each socket’s buffers increased the amount of LOWMEM used. Once LOWMEM was full the kernel (instead of simply panicking) found the user process responsible for the usage and promptly killed it so it could continue to function.

On a standard EC2 Small, the configuration is such that the LOWMEM is around 717MB and the rest is “given” to the user. The kernel is smart about reallocating LOWMEM for the user, but not the other way around. The assumption is that the kernel will use very little ram, or at least a predictable finite amount, and the user should be allowed to go crazy. What we needed with our socket server was just the opposite. We needed the kernel to use all the ram it needed—our Java server rarely uses above a few hundred MB.

(简单的说：在32位Linux机器上，服务器端运行一段时间就被killed掉，由于Linux内核可用内存不够,即低端内存不够，大约只有717MB可用，

32位Linux系统下，内核空间极限896M)。

(For an in-depth rundown, take a look at High Memory In The Linux Kernel)

On a 32-bit system the kernel-addressable RAM space is 4GB. Making sure the proper space reserved for the kernel is important. But on 64-bit (x86-64) Linux the kernel-addressable space is 64TB (terabytes). At the current state of computing this is effectively limitless, and as such you will not even see LowMem in /proc/meminfo because it is all LOWMEM.

So we created some EC2 Large instances (each of which is 64-bit with 7.5GB of RAM) and ran our tests again, this time without any surprises. The sockets were added happily and the kernel took all the RAM it needed.

Long story short, you can only scale to so many sockets on a 32-bit platform.

Kernel Options

（为了使服务器能接收尽量多的连接，需要调整的内核参数）

Several parameters exist to allow for tuning and tweaking of socket-related parameters. In /etc/sysctl.conf there are a few options we’ve modified.

First is fs.file-max, the maximum file descriptor limit. The default is quite low so this should be adjusted. Be careful if you’re not ready to go super high.

Second, we have the socket buffer parameters net.ipv4.tcp_rmem and net.ipv4.tcp_wmem. These are the buffers for reads and writes respectively. Each requires three integer inputs: min, default, and max. These each correspond to the number of bytes that may be buffered for a socket. Set these low with a tolerant max to reduce the amount of ram used for each socket.

The relevant portions of our config look like this:

fs.file-max = 999999

net.ipv4.tcp_rmem = 4096 4096 16777216

net.ipv4.tcp_wmem = 4096 4096 16777216

Meaning that the kernel allows for 999,999 open file descriptors and each socket buffer has a minimum and default 4096-byte buffer, with a sensible max of 16MB.

We also modified /etc/security/limits.conf to allow for 999,999 open file descriptors for all users.

#<domain> <type> <item> <value>

* - nofile 999999

You may want to look at the manpage for more information.

（作者调整的是文件系统中最大文件数限制，tcp读写buffer大小。如果是单进程，还需要调整每个进程能打开的最大文件数，ulimit -n）

Testing

When testing, we were able to get about 64,000 connections per client by increasing the number of ephemeral ports allowed on both the client and the server.

echo "1024 65535" > /proc/sys/net/ipv4/ip_local_port_range

This effectively allows every ephemeral port above 1024 be used instead of the default, which is a much lower (and typically more sane) default.

（调整端口范围，必须在1024之后。）

The 64k Connection Myth

It’s a common misconception that you can only accept 64,000 connections per IP address and the only way around it is to add more IPs. This is absolutely false.

The misconception begins with the premise that there are only so many ephemeral ports per IP. The truth is that the limit is based on the IP pair, or said another way, the client and server IPs together. A single client IP can connect to a server IP 64,000 times and so can another client IP.

Were this myth true it would be a significant and easy-to-exploit DDoS vector.

（常见误区）

Scaling for Everyone

When we set out to establish half a million connections on a single server we were diving deep into water that wasn’t well documented. Sure, we know that C10k is relatively trivial, but how about an order of magnitude (and then some) above that?

Fortunately we’ve been able to achieve success without too many serious problems. Hopefully our solutions can help save time for those out there looking to solve the same problems.

///////////////////////////////////////////

摘抄了部分精彩回复

John Kalucki at 12:10 am on October 1, 2010

If you lose connectivity to a large fraction of the internet, your sockets will back up and you might exhaust memory and cause the TCP stack to freeze. Setting the aggregate wmem demand possible to be only a fraction of RAM avoids this out-of-memory situation at the cost of throughput-per-socket. (正确的设置tcp_wmen，否则网络状况不好的时候，oom)

We’ve experienced just this situation a number of times on the User Streams clusters of the Twitter Streaming API. A route flap, or a LB restart can cause an entire cluster of boxes to lock up if we don’t manage wmem correctly.

You may also want to tune tcp_retries2 down from the default of 15, which is about 2 hours, to something more reasonable. This allows you to reap connections to devices that have been shut off, and thus increase your density of active users per server. Otherwise the number of dead connections per server can, in our use-case, reach as high as 30%. We tuned down to 5 or 6, and the ratio of stale established connections is now very low.

(补充tcp_retries2：

How many times to retry before killing alive TCP connection. RFC1122 says that the limit should be longer than 100 sec. It is too small number. The default value of 15 corresponds to ~ 13 - 30 minutes, depending on RTO.

参考tcp协议栈性能调整系列文章)

本文链接

Linux tcp socket相关参数设置

2011-04-03T04:38:00Z

内容主要从网络上摘抄的，稍作了一下整理

来源1：

http://www.cnblogs.com/alli/archive/2011/01/11/1932599.html

1.进程打开文件数限制

ulimit –n ==》 ulimit –n 1000000

这表示当前用户的每个进程最多允许同时打开1024个文件，这1024个文件中还得除去每个进程必然打开的标准输入，标准输出，标准错误，服务器监听socket，进程间通讯的unix域socket等文件，那么剩下的可用于客户端socket连接的文件数就只有大概1024-10=1014个左右。也就是说缺省情况下，基于Linux的通讯程序最多允许同时1014个TCP并发连接。

典型现象：当connect,accpet 1000个左右连接后开始失败。

2、网络内核对TCP连接的有关限制

2.1 端口号范围限制

net.ipv4.ip_local_port_range = 1025 65000

到同一个主机的连接，超过端口用完后就会失败，这就是我们经常在网上看到的，说客户端最多只能向服务器发起6万多个连接。

2.2 iptable netfilter

第二种无法建立TCP连接的原因可能是因为Linux网络内核的IP_TABLE防火墙对最大跟踪的TCP连接数有限制。此时程序会表现为在connect()调用中阻塞，如同死机，如果用tcpdump工具监视网络，也会发现根本没有TCP连接时客户端发SYN包的网络流量。

net.ipv4.ip_conntrack_max = 10240

3.进程号

cat /proc/sys/kernel/pid_max

But you can change that too; on a 32-bit machine that'd be 2**22 as an absolute upper limit,

so:% sudo bash -c 'echo 4194303 > /proc/sys/kernel/pid_max'

4. tcp 协议栈参数

待续...

本文链接

对Linux下socket限制的理解

2011-04-03T04:25:00Z

最近在网络上看到一个关于Linux下socket数量限制相关的讨论，主要讨论在linux下，是否能支持10甚至百万个tcp并发连接。

讨论 1：在32为系统下，最大连接数的极限值是多少？

从系统内存的角度考虑，32位系统最大支持4G内存，内核空间为1G。如果每个socket占用的内存为C,那么最大连接数为：

maxconn = 1GB/C

如何分析每个socket占用的内核空间内存呢？分析socket占用内存的组成部分：文件系统inode + socket结构 + tcp windows。

tcp windows的内存实际上就是挂在sk结构下的skb队列，skb是真正存储数据的地方，也就是消耗内存的地方。如果CPU的响应速度足够快，网络响应也很快，

那么skb队列只需要保留1个就可以，假定为1.5K。

文件系统inode + socket结构是固定大小的，大约为1K。

理想情况下：每个socket占用的内存为2~3K。以2K计算，内核空间实际可用内存约在800M，那么800M/2K = 400K.

粗略估计，Linux下，32位系统，能支持40万tcp并发连接差不多就到极限了。

分析过程是否有问题，还望高手指教。

HAProxy是通过socket的方式实现的，一直没有找到关于最大并发连接的测试结果，在32位Linux上，最大值估计在20万并发连接左右。

本文链接

Linux 网卡如何支持TSO GSO指南

2011-04-03T03:25:00Z

来源链接：http://gavin1992.gotoip2.com/papperdetails_tech.php?pid=11

1.什么是TSO GSO

TSO是tcp segment offload的缩写，GSO是 generic segmentation offload 的缩写。

详细解释参看http://en.wikipedia.org/wiki/Large_segment_offload

对TSO的简单理解就是：

比如：我们要用汽车把3000本书送到另一个城市，每趟车只能装下1000本书，

那么我们就要书分成3次来发。如何把3000本书分成3份的事情是我们做的，汽车司机只负责运输。

TSO的概念就是：我们把3000本书一起给司机，由他去负责拆分的事情，这样我们就有更多的时间处理其他事情。

对应到计算机系统中，“我们”就是CPU，“司机”就是网卡。

在网络系统中，发送tcp数据之前，CPU需要根据MTU（一般为1500）来将数据放到多个包中发送，对每个数据包都要添加ip头，tcp头，分别计算IP校验和，TCP校验和。如果有了支持TSO的网卡，CPU可以直接将要发送的大数据发送到网卡上，由网卡硬件去负责分片和计算校验和。

2. TSO GSO网卡驱动与系统的接口：

步骤1. 设置支持TSO support flag，同时需要支持SG

netdev->features |= NETIF_F_TSO;

netdev->features |= NETIF_F_SG | NETIF_F_IP_CSUM;

步骤2 设置GSO最大值

netdev ->gso_max_size = 8*1024; //网卡支持的gso size，通知系统每个tcp数据块的最大长度。

//TCP的窗口大小最大为64K，

步骤3：发送函数需要处理skb数据。

支持tso的skb数据存储格式如下：

第一块数据存储在skb的data->tail之间，其他分块存储在skb_shinfo(skb)->frags中。

代码示例如下：

int xmit_support_sg_tso(struct sk_buff *skb)
{
    size = (skb->tail - skb->data);  // the first fragment is stored in the skb.
    for (f = 0; f < skb_shinfo(skb)->nr_frags; f++)    {
            size += skb_shinfo(skb)->frags[f].size; ////other frags .
    }

    memcpy(dbg_send_queue, skb->data,   skb->tail - skb->data);   //real first frag
    for (f = 0; f < skb_shinfo(skb)->nr_frags; f++)  //process frags.
    {
            struct skb_frag_struct *frag;
            int f_offset = 0;
            int f_len = 0;
            char *addr;

            frag = &skb_shinfo(skb)->frags[f];
            f_len = frag->size;
            f_offset = frag->page_offset;

            addr = (char *)page_address(frag->page) ;  //change page addr to virt addr.

           memcpy(dbg_send_queue, addr + f_offset,         f_len);
           offset += f_len;
    }
}

步骤4：支持 ethtool

static struct ethtool_ops comNIC_ethtool_ops = {

.get_settings = netdev_get_settings,

.set_settings = netdev_set_settings,

.get_drvinfo = netdev_get_drvinfo,

.get_link = netdev_get_link,

.get_rx_csum = cmb_ethtool_op_get_rx_csum,

.set_rx_csum = cmb_ethtool_op_set_rx_csum,

.get_tx_csum = cmb_ethtool_op_get_tx_csum,

.set_tx_csum = cmb_ethtool_op_set_tx_csum,

//.set_sg = ethtool_op_set_sg,

#ifdef NETIF_F_TSO

.get_tso = cmb_ethtool_op_get_tso,

.set_tso = cmb_ethtool_op_set_tso,

#endif

}

命令行检验：

查看是否支持tso gso等：

ethtool -k comnic0

设置tso gso打开和关闭

ethtool -K comnic0 tso off

ethtool -K comnic0 gso off

-----------------------------------驱动对tso gso的支持完成-----------------------------------------------

ps：1.支持tso需要同时声明支持scattle / gather, 因为skb的分片数据不是存储在一个连续的地址上。当然：网卡硬件可以不支持scattle/gather这种DMA方式。

2. 需要同时支持硬件校验和。

本文链接

查看Linux内核丢包信息

2010-12-13T08:10:00Z

#cat /proc/net/softnet_stat

0000001f 00000000 00000001 00000000 00000000 00000000 00000000 00000000 00000000
00000016 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000018 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00001143 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

解释：

4行代表有4个cpu。

第一列为该CPU所接收到的所有数据包,

第二列为该CPU缺省queue满的时候, 所删除的包的个数,

(没有统计对于使用NAPI的adapter, 由于ring 满而导致删除的包，在网卡驱动中统计),

第三列表示time_squeeze, 就是说,一次的软中断的触发还不能处理完目前已经接收的数据,因而要设置下轮软中断,time_squeeze 就表示设置的次数.

最后一列，cpu冲突次数。

内核代码：

static int softnet_seq_show(struct seq_file *seq, void *v)
{
struct netif_rx_stats *s = v;

seq_printf(seq, "%08x %08x %08x %08x %08x %08x %08x %08x %08x\n",
s->total, s->dropped, s->time_squeeze, 0,
0, 0, 0, 0, /* was fastroute */
s->cpu_collision );
return 0;
}

include/linux/netdevice.h:DECLARE_PER_CPU(struct netif_rx_stats,
netdev_rx_stat);

struct netif_rx_stats
{
unsigned total;
unsigned dropped;
unsigned time_squeeze;
unsigned cpu_collision;
};

int netif_rx(struct sk_buff *skb)
{
   struct softnet_data *queue;
   unsigned long flags;

   /* if netpoll wants it, pretend we never saw it */
   if (netpoll_rx(skb))
       return NET_RX_DROP;

   if (!skb->tstamp.off_sec)
       net_timestamp(skb);

   /*
   * The code is rearranged so that the path is the most
   * short when CPU is congested, but is still operating.
   */
   local_irq_save(flags);
   queue = &__get_cpu_var(softnet_data);

   __get_cpu_var(netdev_rx_stat).total++;
   if (queue->input_pkt_queue.qlen <= netdev_max_backlog) {
       if (queue->input_pkt_queue.qlen) {
enqueue:
           dev_hold(skb->dev);
           __skb_queue_tail(&queue->input_pkt_queue, skb);
           local_irq_restore(flags);
           return NET_RX_SUCCESS;
       }

       netif_rx_schedule(&queue->backlog_dev);
       goto enqueue;
   }

   __get_cpu_var(netdev_rx_stat).dropped++;
   local_irq_restore(flags);

   kfree_skb(skb);
   return NET_RX_DROP;
}

本文链接

Linux 内核性能工具

2010-11-17T07:24:00Z

Oprofile：

http://oprofile.sourceforge.net/docs/

opcontrol --list-events

本文链接

struct sk_buff

2010-11-11T05:36:00Z

info:

本文链接

转载 google hack

2010-10-09T01:26:00Z

转载自月光博客 [ http://www.williamlong.info/ ]

intext:这个就是把网页中的正文内容中的某个字符做为搜索条件.例如在google里输入:intext:动网.将返回所有在网页正文部分包含"动网"的网页.allintext:使用方法和intext类似.

intitle:
和上面那个intext差不多,搜索网页标题中是否有我们所要找的字符.例如搜索:intitle:安全天使.将返回所有网页标题中包含"安全天使"的网页.同理allintitle:也同intitle类似.

cache:
搜索google里关于某些内容的缓存,有时候也许能找到一些好东西哦.

define:
搜索某个词语的定义,搜索:define:hacker,将返回关于hacker的定义.

filetype:
这个我要重点推荐一下,无论是撒网式攻击还是我们后面要说的对特定目标进行信息收集都需要用到这个.搜索指定类型的文件.例如输入:filetype: doc.将返回所有以doc结尾的文件URL.当然如果你找.bak、.mdb或.inc也是可以的,获得的信息也许会更丰富:)

info:
查找指定站点的一些基本信息.

inurl:
搜索我们指定的字符是否存在于URL中.例如输入:inurl:admin,将返回N个类似于这样的连接:http://www.xxx.com/xxx/admin,用来找管理员登陆的URL不错.allinurl也同inurl类似,可指定多个字符.

link:
例如搜索:inurl:www.4ngel.net可以返回所有和www.4ngel.net做了链接的URL.

site:
这个也很有用,例如:site:www.4ngel.net.将返回所有和4ngel.net这个站有关的URL.

对了还有一些操作符也是很有用的:

+ 把google可能忽略的字列如查询范围
- 把某个字忽略
~ 同意词
. 单一的通配符
* 通配符，可代表多个字母
"" 精确查询

下面开始说说实际应用(我个人还是比较习惯用google.com,以下内容均在google上搜索),对于一个居心叵测的攻击者来说,可能他最感兴趣的就是密码文件了.而google正因为其强大的搜索能力往往会把一些敏感信息透露给他们.用google搜索以下内容:

intitle:"index of" etc
intitle:"Index of" .sh_history
intitle:"Index of" .bash_history
intitle:"index of" passwd
intitle:"index of" people.lst
intitle:"index of" pwd.db
intitle:"index of" etc/shadow
intitle:"index of" spwd
intitle:"index of" master.passwd
intitle:"index of" htpasswd
"# -FrontPage-" inurl:service.pwd

有时候因为各种各样的原因一些重要的密码文件被毫无保护的暴露在网络上,如果被别有用心的人获得,那么危害是很大的.下面是我找到的一个FreeBSD系统的passwd文件(我已做过处理):

同样可以用google来搜索一些具有漏洞的程序,例如ZeroBoard前段时间发现个文件代码泄露漏洞,我们可以用google来找网上使用这套程序的站点:
intext:ZeroBoard filetype:php
或者使用:
inurl:outlogin.php?_zb_path= site:.jp
来寻找我们所需要的页面.phpmyadmin是一套功能强大的数据库操作软件,一些站点由于配置失误,导致我们可以不使用密码直接对phpmyadmin进行操作.我们可以用google搜索存在这样漏洞的程序URL:
intitle:phpmyadmin intext:Create new database

还记得http://www.xxx.com/_vti_bin/..%5C..%5C..%5C..%5C..%5C../winnt/system32/cmd.exe?dir吗?用google找找，你也许还可以找到很多古董级的机器。同样我们可以用这个找找有其他cgi漏洞的页面。
allinurl：winnt system32

前面我们已经简单的说过可以用google来搜索数据库文件,用上一些语法来精确查找能够获得更多东西(access的数据库,mssql、mysql的连接文件等等).举个例子示例一下:
allinurl:bbs data
filetype:mdb inurl:database
filetype:inc conn
inurl:data filetype:mdb
intitle:"index of" data //在一些配置不正确的apache+win32的服务器上经常出现这种情况
和上面的原理一样,我们还可以用google来找后台,方法就略了,举一反三即可,毕竟我写这篇文章的目的是让大家了解google hacking,而不是让你用google去破坏.安全是把双刃剑,关键在于你如何去运用.

利用google完全是可以对一个站点进行信息收集和渗透的，下面我们用google对特定站点进行一次测试。www.xxxx.com是全国著名大学之一，一次偶然的机会我决定对其站点进行一次测试(文中所涉及该学校的信息均已经过处理，请勿对号入座:).
    首先用google先看这个站点的一些基本情况(一些细节部分就略去了):
site:xxxx.com
    从返回的信息中，找到几个该校的几个系院的域名：
http://a1.xxxx.com
http://a2.xxxx.com
http://a3.xxxx.com
http://a4.xxxx.com
    顺便ping了一下，应该是在不同的服务器.(想想我们学校就那一台可怜的web服务器，大学就是有钱，汗一个)。学校一般都会有不少好的资料，先看看有什么好东西没:
site:xxxx.com filetype:doc
得到N个不错的doc。先找找网站的管理后台地址：
site:xxxx.com intext:管理
site:xxxx.com inurl:login
site:xxxx.com intitle:管理
超过获得2个管理后台地址：
http://a2.xxxx.com/sys/admin_login.asp
http://a3.xxxx.com:88/_admin/login_in.asp
还算不错，看看服务器上跑的是什么程序：
site:a2.xxxx.com filetype:asp
site:a2.xxxx.com filetype:php
site:a2.xxxx.com filetype:aspx
site:a3.xxxx.com filetype:asp
site:.......
......
a2服务器用的应该是IIS，上面用的是asp的整站程序，还有一个php的论坛
a3服务器也是IIS，aspx+asp。web程序都应该是自己开发的。有论坛那就看看能不能遇见什么公共的FTP帐号什么的：
site:a2.xxxx.com intext:ftp://*:*
没找到什么有价值的东西。再看看有没有上传一类的漏洞：
site:a2.xxxx.com inurl:file
site:a3.xxxx.com inurl:load
在a2上发现一个上传文件的页面：
http://a2.xxxx.com/sys/uploadfile.asp
用IE看了一下，没权限访问。试试注射，
site:a2.xxxx.com filetype:asp
得到N个asp页面的地址，体力活就让软件做吧，这套程序明显没有对注射做什么防范，dbowner权限，虽然不高但已足矣，back a shell我不太喜欢，而且看起来数据库的个头就不小，直接把web管理员的密码暴出来再说，MD5加密过。一般学校的站点的密码都比较有规律，通常都是域名+电话一类的变形，用google搞定吧。
site:xxxx.com    //得到N个二级域名
site:xxxx.com intext:*@xxxx.com //得到N个邮件地址，还有邮箱的主人的名字什么的
site:xxxx.com intext:电话     //N个电话
把什么的信息做个字典吧，挂上慢慢跑。过了一段时间就跑出4个帐号，2个是学生会的，1个管理员，还有一个可能是老师的帐号。登陆上去：
name：网站管理员
pass：a2xxxx7619    //说了吧，就是域名+4个数字
要再怎么提权那就不属于本文讨论访问了，呵呵，到此为止。

关于google hacking的防范：
以前我们站的晓风·残月写过一篇躲避google的文章，原理就是通过在站点根目录下建立一个robots.txt以避免网络机器人获得一些敏感的信息，具体大家看原文章：http://www.4ngel.net/article/26.htm。
不过这种方法我个人不推荐，有点此地无银三百两的味道。简单一点的方法就是上google把自己站点的一些信息删除掉，访问这个URL：
http://www.google.com/remove.html
前几天看见又有人讨论用程序来欺骗robot的方法，我觉得可以试试：
代码如下：
<?php

if (strstr($_SERVER['HTTP_USER_AGENT'], "Googlebot"))
{
header("HTTP/1.1 301");
header("Location: http://www.google.com");
}

asp的：
<%
if instr(Request.Servervariables("HTTP_USER_AGENT"),"Googlebot") then
response.redirect("http://www.google.com")
end if

后记
这段时间在国外的一些google hack的研究站点看了看，其实也都差不多是一些基本语法的灵活运用，或者配合某个脚本漏洞，主要还是靠个人的灵活思维。国外对于google hack方面的防范也并不是很多，所以大家还是点到为止，不要去搞破坏拉，呵呵。对于一些在win上跑
apache的网管们应该多注意一下这方面，一个intitle:index of就差不多都出来了:)

webcam:

“inurl:"ViewerFrame?Mode="”，
“inurl: "MultiCameraFrame?Mode="”，
“intitlE:"Live View / - AXIS"”，
“inurl:"axis-cgi/mjpg"”，
“intext:"MOBOTIX M1"
intext:"Open Menu"”或者
“inurl:"view/index.shtml" ”

作者：sniper　出处：http://www.4ngel.net/article/51.htm

本文链接

libnids 相关

2010-08-23T03:52:00Z

最近听过应用层的朋友用的是libnids库，因此大致看了一下。

1.libnids库完全是应用层的函数库，基本上没有内核相关的操作。

2.libnids底层收包用的是libpcap函数，几个主要的libpcap函数接口是：

pcap_open_live(pcap_open_offline) --> nids_init

pcap_loop --> nids_run

pcap_next --> nids_next

pcap_dispatch --> nids_dispatch

3.报文处理回调函数封装为：

nids_pcap_handler

4.libnids处理ip分片，tcp会话重组。libnids号称的稳定性据说是因为“代码拷贝自Linux内核，因此与内核协议栈有一样的稳定性”。

TCP会话重组部分似乎比内核中的代码简单，可能是因为buffer处理部分比较简单。

//==================

5. 用户注册回调函数：

nids_register_ip_frag （接收未进行ip分片重组的原始报文）

nids_register_ip

nids_register_tcp

nids_register_udp

分别存储在：ip_frag_procs，ip_procs，tcp_procs，udp_procs的item指针中。

6.在ndis_init的时候，首先调用init_procs，

a.在ip_frag_procs中注册默认处理函数：gen_ip_frag_proc，调用ip_defrag_stub -->ip_defrag 进行ip分片重组。

b.在ip_procs中注册默认的处理函数：gen_ip_proc， gen_ip_proc函数根据报文类型处理tcp，dup，icmp包。

7.tcp会话重组，待续。

本文链接

C转义字符问题

2010-07-20T12:02:00Z

今天遇到了一个C语言的问题，以前没注意到：

http://wordaligned.org/articles/octal-literals

Very large Hex value in a String Literal

std::string s = "ABC\x7FDEF";

Here, the "DEF" characters are valid hexadecimal and therefore become part of the number we’re embedding; so we’ve tried to put the hex number 7FDEF into a byte. If we’re lucky our compiler will warn us:

warning: hex escape sequence out of range

If we’re unlucky or if we don’t act on the warning, the result is implementation defined. In any case, it’s certainly not what we wanted. Of course, embedded octal escape sequences suffer from the exact same problem if succeeded by one of the letters "0" - "7".

The workaround is simple:

Hex 7F in a String Literal

std::string s = "ABC\x7F" "DEF";

or even:

Hex 7F in a String Literal

std::string s = "ABC" "\x7F" "DEF";

In other words, even this use of octal values is of limited practical use.

以后要用正确的方式使用8，16进制转义。

本文链接