博客园_回头再说坚强2002的博客

[Erlang 0035] Erlang SMP

2012-02-01T04:36:00Z

Erlang SMP

Erlang SMP (Symmetrical Multi Processor)在97-98年左右开始,项目按照先跑起来再优化的开发策略("First make it work, then measure, then optimize".),在2006年R11B发布了第一个稳定版本;

原理

没有SMP支持的Erlang VM只有一个Scheduler运行于主线程;Scheduler从运行队列中取出需要处理的Erlang进程和IO-job;由于只有一个Scheduler没有必要对数据加锁;见下图:

有SMP支持的Erlang VM 可以有1~1024个Scheduler,每一个Scheduler都会运行于一个独立的操作系统线程;操作系统决定它是否要在不同的核上运行.由于多个Scheduler就要对数据加锁,一个Erlang进程可能前后被多个Scheduler调度;

性能

下面这个数据比较老了(2008年1月),但是还可以做参考:

Measurements from a real telecom product showed a 1.7 speed improvement between a single and a dual core system.

The SMP VM with only one scheduler is slightly slower (10%) than the non SMP VM.
This is because the SMP VM need to use locks for all shared datastructures. But as long as there are no lock-conflicts the overhead caused by locking is not that high (it is the lock conflicts that takes time).
This explains why it in some cases can be more efficient to run several SMP VM's with one scheduler each instead on one SMP VM with several schedulers. Of course the running of several VM's require that the application can run in many parallel tasks which has no or very little communication with each other.

应用

从OTP R12B开始只要操作系统告知当前是多CPU(多核),SMP就会自动开启并设定Scheduler个数和CPU或核的数量一致;我们启动一下看看:

Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9  (abort with ^G)
1>

检查一下服务器CPU信息

# grep "model name" /proc/cpuinfo | cut -f2 -d:
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz
Intel(R) Xeon(R) CPU           E5506 @ 2.13GHz

可以看到8核的机器启动的时候自动开启了smp:8:8,这两个数字是什么意思?

+S Schedulers:SchedulerOnline

Sets the amount of scheduler threads to create and scheduler threads to set online when SMP support has been enabled. Valid range for both values are 1-1024. If the Erlang runtime system is able to determine the amount of logical processors configured and logical processors available, Schedulers will default to logical processors configured, and SchedulersOnline will default to logical processors available; otherwise, the default values will be 1. Schedulers may be omitted if :SchedulerOnline is not and vice versa. The amount of schedulers online can be changed at run time via erlang:system_flag(schedulers_online, SchedulersOnline).

注意:

如果设定的数量超出CPU数或者核数并不能得到什么好处
有的操作系统可以使用tasket之类的命令绑定CPU,Erlang VM只会检测可用的CPU数,因为绑定这事可能在任何时刻发生;SchedulersOnline 就是实际可用的CPU数或核数;
运行时是可以调整该参数的

如果要关闭SMP 可以使用下面的启动参数:

-smp [enable|auto|disable]
-smp enable and -smp starts the Erlang runtime system with SMP support enabled. This may fail if no runtime system with SMP support is available. -smp auto starts the Erlang runtime system with SMP support enabled if it is available and more than one logical processor are detected. -smp disable starts a runtime system without SMP support. By default -smp auto will be used unless a conflicting parameter has been passed, then -smp disable will be used. Currently only the -hybrid parameter conflicts with -smp auto.

总结

SMP使用操作系统线程实现多个调度器,利用了多核多CPU的优势并把实现细节对开发者隐藏起来, 可以不修改代码甚至不用重新编译就可以使用;使用SMP可以有非常灵活的启动选择和运行时调整的方法入口;

相关资料:

[1] 本文截图来自Ericsson的一篇文档:Inside the Erlang VM with focus on SMP

[2] Some facts about Erlang and SMP http://erlang.2086793.n4.nabble.com/Some-facts-about-Erlang-and-SMP-td2108770.html

本文链接

[Erlang 0034] Erlang iolist

2012-01-31T09:58:00Z

问题的缘起是芒果在使用mochiweb的过程中遇到一个异常，在google的讨论组中我找到了同样的问题：

=ERROR REPORT==== 7-Apr-2011::18:58:22 === 
"web request failed" 
path: "cfsp/entity" 
type: error 
what: badarg 
trace: [{erlang,iolist_size, 
[[...]]}, 
{mochiweb_request,respond,2}, 
{rest_server_web,loop,1}, 
{mochiweb_http,headers,5}, 
{proc_lib,init_p_do_apply,3}]

提问者遇到这个异常后判断是文档超长造成的，bob在下面的回复首先否定了这个猜测，并把关注点放在了trace信息中明确提示出来的iolist异常上面，他的回复：

I don't think it has anything to do with the size of your document,

your code is somehow returning a value that is not an iolist. Perhaps there is an atom in there, or an integer outside of 0..255 (I guess this is more likely, I don't know xmerl output very well).

1> iolist_size([256]). 
** exception error: bad argument 
in function iolist_size/1 
called as iolist_size([256])

You probably want UTF-8, so unicode:characters_to_binary(xmerl:export_simple(Res,xmerl_xml)) is my

guess at what you really want to be doing for the output.

-bob

详情点击：http://groups.google.com/group/mochiweb/browse_thread/thread/f67abc113b338bfe?pli=1

按照这个提示给芒果，果然就把问题解决掉了；问题到这里还不能结束，追问一下，Erlang的List和IOList有什么区别？

iolist定义

在erlang官方文档中iolist描述甚少,不过还是可以找到定义:

iolist()     maybe_improper_list(char() | binary() | iolist(), binary() | [])
maybe_improper_list()     maybe_improper_list(any(), any())
byte()     0..255
char()     0..16#10ffffmaybe_improper_list(T)     maybe_improper_list(T, any())

注意iolist相关的两个函数,他们接收的参数还可以是binary

iolist_size(Item) -> integer() >= 0

Types:Item = iolist() | binary() 

iolist_to_binary(IoListOrBinary) -> binary()

Types:IoListOrBinary = iolist() | binary()

官方文档地址:http://www.erlang.org/doc/reference_manual/typespec.html

我们动手测试一下:

Eshell V5.9  (abort with ^G)
1> iolist_size([]).
0
2> iolist_size([<<"anc">>]).
3
3> iolist_size([12,<<"anc">>]).
4
4> iolist_size([12,<<"anc">>,23]).
5
5> iolist_size([12,<<"anc">>,23,<<"king">>]).
9
6> iolist_size([12,<<"anc">>,23,<<"king">>,[23,34,<<"test">>]]).
15
7> iolist_size(<<"abc">>).
3
8> iolist_size(<<>>).
0
9> iolist_size([1234]).
** exception error: bad argument
     in function  iolist_size/1
        called as iolist_size([1234])
10> iolist_size([<<1:1>>]).
** exception error: bad argument
     in function  iolist_size/1
        called as iolist_size([<<1:1>>])
11> iolist_size( [12,23,"abc",<<abc>>]).
** exception error: bad argument
12> iolist_size( [12,23,<<abc>>]).
** exception error: bad argument
13> iolist_size( [12,23,"abc",<<"abc">>]).
8
14>  L=[$H, $e, [$l, <<"lo">>, " "], [[["W","o"], <<"rl">>]] | [<<"d">>]].
[72,101,[108,<<"lo">>," "],[[["W","o"],<<"rl">>]],<<"d">>]

iolist适用的场景?

首先能够找到的是mryufeng的这篇《iolist跟list有什么区别？》 http://mryufeng.iteye.com/blog/634867

这篇文章分析源码得到了iolist数据结构的定义,并在解释了iolist的作用:

Iolist的作用是用于往port送数据的时候.由于底层的系统调用如writev支持向量写, 就避免了无谓的iolist_to_binary这样的扁平话操作, 避免了内存拷贝,极大的提高了效率.建议多用.

这个是什么意思呢? 在Learn you some Erlang站点上<<Buckets of Sockets>>一文的开篇我找到了答案:

A = [a]
B = [b|A] = [b,a]
C = [c|B] = [c,b,a]
In the case of prepending, as above, whatever is held into A or B or C never needs to be rewritten. The representation of C can be seen as either [c,b,a], [c|B] or [c,|[b|[a]]], among others. In the last case, you can see that the shape of A is the same at the end of the list as when it was declared. Similarly for B. Here's how it looks with appending:

A = [a]
B = A ++ [b] = [a] ++ [b] = [a|[b]]
C = B ++ [c] = [a|[b]] ++ [c] = [a|[b|[c]]]
Do you see all that rewriting? When we create B, we have to rewrite A. When we write C, we have to rewrite B (including the [a|...] part it contains). If we were to add D in a similar manner, we would need to rewrite C. Over long strings, this becomes way too inefficient, and it creates a lot of garbage left to be cleaned up by the Erlang VM.

With binaries, things are not exactly as bad:

A = <<"a">>
B = <<A/binary, "b">> = <<"ab">>
C = <<B/binary, "c">> = <<"abc">>
In this case, binaries know their own length and data can be joined in constant time. That's good, much better than lists. They're also more compact. For these reasons, we'll often try to stick to binaries when using text in the future.

There are a few downsides, however. Binaries were meant to handle things in certain ways, and there is still a cost to modifying binaries, splitting them, etc. Moreover, sometimes we'll work with code that uses strings, binaries, and individual characters interchangeably. Constantly converting between types would be a hassle.

In these cases, IO lists are our saviour. IO lists are a weird type of data structure. They are lists of either bytes (integers from 0 to 255), binaries, or other IO lists. This means that functions that accept IO lists can accept items such as [$H, $e, [$l, <<"lo">>, " "], [[["W","o"], <<"rl">>]] | [<<"d">>]]. When this happens, the Erlang VM will just flatten the list as it needs to do it to obtain the sequence of characters Hello World.

What are the functions that accept such IO Lists? Most of the functions that have to do with outputting data do. Any function from the io module, file module, TCP and UDP sockets will be able to handle them. Some library functions, such as some coming from the unicode module and all of the functions from the re (for regular expressions) module will also handle them, to name a few.

Try the previous Hello World IO List in the shell with io:format("~s~n", [IoList]) just to see. It should work without a problem.

All in all, they're a pretty clever way of building strings to avoid the problems of immutable data structures when it comes to dynamically building content to be output.

简单说明一下上面的内容:

|->如果是在List头部追加内容是非常快速的,但是在List尾部追加内容就要进行遍历

-> 使用binary数据可以在常量时间内完成尾部追加,但是问题:①修改和split存在消耗 ;②字符和二进制数据的常量转换

-> iolist对这种数据混搭有一个较好的支持,Erlang VM会将list平铺,可以使用io:format来检验各种数据构成的iolist输出之后的结果

-> 总结 iolist是单次赋值约束下,动态构建字符串内容输出的好方法;

我们可以通过erlc +\'to_core\' M.erl 的方法(参见:[Erlang 0029] Erlang Inline编译)查看一下iolist的 Core Erlang表示:

在Core Erlang中List = [1,2,3,4,5,6,7,8,9],会被表示为:[1|[2|[3|[4|[5|[6|[7|[8|[9]]]]]]]]]

看下

L=[$H, $e, [$l, <<"lo">>, " "], [[["W","o"], <<"rl">>]] | [<<"d">>]],
  iolist_size(L)

转换为:

do  %% Line 14
            call 'erlang':'iolist_size'
                ([72|[101|[[108|[#{#<108>(8,1,'integer',['unsigned'|['big']]),
                                   #<111>(8,1,'integer',['unsigned'|['big']])}#|[[32]]]]|[[[[[87]|[[111]]]|[#{#<114>(8,1,'integer',['unsigned'|['big']]),
                                                                                                              #<108>(8,1,'integer',['unsigned'|['big']])}#]]]|[#{#<100>(8,1,'integer',['unsigned'|['big']])}#]]]]])

相关阅读

Stackoverflow上有人提到了同样的问题:

Ports, external or linked-in, accept something called io-lists for sending data to them. An io-list is a binary or a (possibly deep) list of binaries or integers in the range 0..255.
This means that rather than concatenating two lists before sending them to a port, one can just send them as two items in a list. So instead of
"foo" ++ "bar"
one do
["foo", "bar"]
In this example it is of course of miniscule difference. But the iolist in itself allows for convenient programming when creating output data. io_lib:format/2,3 itself returns an io list for example.
The function erlang:list_to_binary/1 accepts io lists, but now we have erlang:iolist_to_binary/1 which convey the intention better. There is also an erlang:iolist_size/1.
Best of all, since files and sockets are implemented as ports, you can send iolists to them. No need to flatten or append.

还有这一篇:A Ramble Through Erlang IO Lists http://prog21.dadgum.com/70.html

The IO List is a handy data type in Erlang, but not one that's often discussed in tutorials. It's any binary. Or any list containing integers between 0 and 255. Or any arbitrarily nested list containing either of those two things. Like this:
[10, 20, "hello", <<"hello",65>>, [<<1,2,3>>, 0, 255]]
The key to IO lists is that you never flatten them. They get passed directly into low-level runtime functions (such as file:write_file), and the flattening happens without eating up any space in your Erlang process. Take advantage of that! Instead of appending values to lists, use nesting instead. For example, here's a function to put a string in quotes:
quote(String) -> $" ++ String ++ $".
If you're working with IO lists, you can avoid the append operations completely (and the second "++" above results in an entirely new version of String being created). This version uses nesting instead:
quote(String) -> [$", String, $"].
This creates three list elements no matter how long the initial string is. The first version creates length(String) + 2 elements. It's also easy to go backward and un-quote the string: just take the second list element. Once you get used to nesting you can avoid most append operations completely.

One thing that nested list trick is handy for is manipulating filenames. Want to add a directory name and ".png" extension to a filename? Just do this:
[Directory, $/, Filename, ".png"]
Unfortunately, filenames in the file module are not true IO lists. You can pass in deep lists, but they get flattened by an Erlang function (file:file_name/1), not the runtime system. That means you can still dodge appending lists in your own code, but things aren't as efficient behind the scenes as they could be. And "deep lists" in this case meansonly lists, not binaries. Strangely, these deep lists can also contain atoms, which get expanded via atom_to_list.

Ideally filenames would be IO lists, but for compatibility reasons there's still the need to support atoms in filenames. That brings up an interesting idea: why not allow atoms as part of the general IO list specification? It makes sense, as the runtime system has access to the atom table, and there's a simple correspondence between an atom and how it gets encoded in a binary; 'atom' is treated the same as "atom". I find I'm often calling atom_to_list before sending data to external ports, and that would no longer be necessary.

总结

iolist是单次赋值约束下,避免了字符串和二进制数据的转换,是动态构建字符串内容输出的好方法;

本文链接

[Erlang 0033] 接入Erlang控制台的几种方法

2012-01-18T07:03:00Z

在window中调试的时候我们可以通过启动多个cmd窗口运行Erlang节点,在生产环境中我们需要Erlang服务在Centos服务器上后台运行;这就需要在启动的时候添加启动参数detached来脱离终端:

-detached Starts the Erlang runtime system detached from the system console. Useful for running daemons and backgrounds processes. Implies -noinput.

对于我们自己的服务,即使部署到了生产环境一定要做到"像魔术师的飞刀,出手但并没有脱手",还是需要一些方式进入到Erlang后台进程来做一些工作比如:查看某一个Erlang节点的运行时信息(内存,进程数等),让服务优雅的退出而不是kill进程,或者做一下热更新(参见:[Erlang 0010] Erlang 热更新当然热更新可以使用reloader.erl的方案来简化);一开始的时候服务器比较少,我们采用的是JCL的方式去处理的;

Erlang Shell JCL

JCL是Erlang Shell的一种运行模式,即Job Control Mode (JCL, in which jobs can be started, killed, detached and connected).我们启动两个节点来完成这个操作;
Node_1 添加了-detached选项,启动之后直接在后台运行并没有启动Shell
erl -setcookie abc -name node_1@192.168.1.123 -detached
Node_2 使用了和Node_1相同的cookie,启动之后进入Erlang Shell界面
erl -setcookie abc -name node_2@192.168.1.123
下面我们开始在node_2@192.168.1.123演练JCL:

Eshell V5.9  (abort with ^G)
(node_2@192.168.1.123)1> node().    %当前这是在node_2
'node_2@192.168.1.123'
(node_2@192.168.1.123)2>                %Control + G 进入JCL模式
User switch command
--> h  
  c [nn]            - connect to job
  i [nn]            - interrupt job
  k [nn]            - kill job
  j                 - list all jobs
  s [shell]         - start local shell
  r [node [shell]]  - start remote shell
  q        - quit erlang
  ? | h             - this message
--> r'node_1@192.168.1.123'                           %尝试连接到node_1@192.168.1.123  
--> j
   1  {shell,start,[init]}                                          %列出所有的Job
   2* {'node_1@192.168.1.123',shell,start,[]}
--> c 2                                                               %这里2是job的编号,切换到job 2 
Eshell V5.9  (abort with ^G)
(node_1@192.168.1.123)1> node().                %注意提示符,现在已经是在node_1
'node_1@192.168.1.123'
(node_1@192.168.1.123)2> erlang:now().
{1326,801888,347570}
(node_1@192.168.1.123)3>                             %再一次Control + G
User switch command                                   
--> j                                                                   
   1  {shell,start,[init]}
   2* {'node_1@192.168.1.123',shell,start,[]}
--> c 1                                                               %切换到job 1
(node_2@192.168.1.123)2> node().                %注意提示符,我们已经回到了node_2
'node_2@192.168.1.123'
(node_2@192.168.1.123)3>

这样来来回回切换是不是有点盗梦空间的意思?是不是可以更简单一点,比如直接进入node_1呢?借助-remsh参数就可以做到

看看-remsh的说明恰好是我们需要的:

If you want an Erlang node to have a remote job active from the start (rather than the default local job), you start Erlang with the -remsh flag. Example: erl -sname this_node -remsh other_node@other_host

动手试一下:
erl -setcookie abc -name node_3@192.168.1.123 -remsh node_1@192.168.1.123 %%这样就直接进入了node_1节点
注意:直接进入到了node_1,执行完操作了想要退出怎么办? 你要是在这里执行一下q(). node_1这个节点就直接死掉了;
正确的方法还是Ctrl+G进入JCL模式然后执行q命令退出;使用ps aux|grep node查看一下进程是不是还在

ejabberd网站上提到了这个方法
[1] Attach an Erlang Shell to an Already Running ejabberd Process http://www.ejabberd.im/tricks
[2] Interconnecting Erlang Nodes http://www.ejabberd.im/interconnect-erl-nodes

另一种实用的接入erlang控制台的方法

地址:http://mryufeng.iteye.com/blog/362394 Powered by mryufeng

按照mryufeng老大的方法操练了一番,成功,中间 /usr/local/lib/erlang/bin/start 启动失败,同样是目录和配置文件缺失的问题
mkdir /usr/local/lib/erlang/log
echo "[]." > /usr/local/lib/erlang/releases/R15B/sys.config

Stackoverflow的相关问题: How to create deamon program with erlang? http://stackoverflow.com/questions/5972811/how-to-create-deamon-program-with-erlang

原理见下图:

erl_call

随着服务器增多,登录一台台服务器然后使用JCL去做热更新之类的就有点麻烦了,取而代之的解决方案就是ssh登录到每台服务器然后向该服务器上的Erlang节点发送消息,执行热更新(当然不限于热更新了);

这里有一个很强大的工具可以用:erl_call;erl_call模块的设计目标就是使用Unix Shell脚本与分布式Erlang节点进行交互,erl_call可以启动或者连接到一个分布式的Erlang节点进行简单的方法调用甚至执行表达式序列;erl_call构建在erl_interface之上,通过标准的RPC调用方式与Erlang的rex server交互,并不需要额外的工具支持.注意Erlang节点的RPC服务启动的时候注册的名字是rex http://www.google.com/codesearch#dNzC48WMK6w/lib/kernel/src/rpc.erl&q=rpc%20lang:erlang&type=cs

看一下erl_call的帮助,注意Erlang版本不同erl_call的路径也有所不同:

Usage: /usr/local/lib/erlang/lib/erl_interface-3.7.6/bin/erl_call [-[demqrsv]] [-c Cookie] [-h HiddenName]
            [-x ErlScript] [-a [Mod [Fun [Args]]]]
            (-n Node | -sname Node | -name Node)

where: -a apply(Mod,Fun,Args) (e.g -a 'erlang length [[a,b,c]]'
         -c cookie string; by default read from ~/.erlang.cookie
         -d direct Erlang output to ~/.erl_call.out.<Nodename>
         -e evaluate contents of standard input (e.g echo "X=1,Y=2,{X,Y}."|erl_call -e ...)
         -h specify a name for the erl_call client node
         -m read and compile Erlang module from stdin
         -n name of Erlang node, same as -name
         -name name of Erlang node, expanded to a fully qualified
         -sname name of Erlang node, short form will be used
         -q halt the Erlang node (overrides the -s switch)
         -r use a random name for the erl_call client node
         -s start a new Erlang node if necessary
         -v verbose mode, i.e print some information on stderr
         -x use specified erl start script, default is erl

简单翻译一下:

-a 执行erlang:apply/3,需要提供的就是m f a,m是必须的,f默认值start,a默认值[],注意这个flag只接受一个参数,所以最好通过添加括号把m f a组织起来.

-c 执行cookie值如果没有指定就会使用 ~/.erlang.cookie 文件中定义的cookie,Erlang 节点之间通信必须要有相同的cookie

-d 调试模式;所有的IO都会输出到~/.erl_call.out.Nodename文件,其中NodeName就是我们要处理的erlang节点名;

-e 读一系列的Erlang表达式并返回最后一个表达式的结果,这些表达式用','分隔,'.'结束,这个Erlang Shell是一致的,Control-D组合键结束输入,如果成功返回结果{ok,Result}.

-h HiddenName 隐藏erl_call代表的节点

-m 读入并编译一个module

-n -name -sname 这三个参数三选一,name和n是同样的含义是为了兼容依然保留,sname代码短名称

-q 停止Erlang 节点

-r 为erl_call生一个随机name

-s 必要的话启动一个Erlang的分布式节点,意思是自由第一个调用会启动一个Erlang节点后续的调用就会快很多.

-v 多打印一些冗余信息

-x ErlScript:指定Erlang启动脚本如果不指定就是用erl start-up脚本

比如我们现在要获取节点node_1的内存信息

/usr/local/lib/erlang/lib/erl_interface-3.7.6/bin/erl_call -s -a 'erlang memory ' -name node_1@192.168.1.123 -c abc

实践中遇到两个问题:

问题一:当name里面包含ip地址的时候执行会报错,这个erlang论坛里面有人也提到了:

erl_call can now use an IP address for remote node name http://erlang.org/pipermail/erlang-patches/2010-October/001533.html

erl_call: remove get_hostent http://erlang.org/pipermail/erlang-patches/2010-October/001550.html

我的解决方案就是把name换成了sname并去掉了ip地址就OK了;

问题二:在使用-e参数的时候怎么结束掉输入并查看结果

这个问题的确挺二的,就因为没有仔细阅读文档,结束的时候需要用Ctrl+D,http://www.erlang.org/doc/man/erl_call.html

/usr/local/lib/erlang/lib/erl_interface-3.7.6/bin/erl_call -e -name node_1@192.168.1.123 -c abc

A=1,
B=2,
C=A+B.
%Ctrl + D
{ok, 3}

有了这些方法,我们可以方便的了解和控制运行中的Erlang节点,洋溢着幸福感啊!

各位春节快乐!!!

本文链接

[Erlang 0032] Erlang Binary的内部实现

2012-01-11T16:13:00Z

[Erlang 0014]Erlang垃圾回收机制

[Erlang 0024]Erlang二进制数据处理

上面两篇文章都提到了二进制数据相关的内容,今天阅读了一下官方文档中关于二进制实现的内容,整理笔记于此.先看一张图:

思维导图解释

binary和bitstring内部实现机制相同
Erlang内部有四种二进制类型,两种容器,两种引用
容器有refc binaries 和 heap binaries
refc binaries又可以分成两部分存放在进程堆(process heap)的ProcBin和进程堆以外的二进制对象
ProcBin包含一个二进制数据的元数据信息,包含了二进制数据的位置和引用计数
游离在进程堆之外的二进制对象可以被任意数量的进程和任意数量的ProcBin引用,该对象包含了引用计数器,一旦计数器归零就可以移除掉
所有的ProcBin对象都是链表的一部分,所以GC跟踪它们并在ProcBin消失的时候将应用计数减一
heap binaries 都是小块二进制数据,最大64字节,直接存放在进程堆(process heap),垃圾回收和发送消息都是通过拷贝实现,不需要垃圾回收器做特殊处理
引用类型有两种:sub binaries , match contexts
sub binary是split_binary的时候产生的,sub binary是另外一个二进制数据的部分应用(refc 或者 heap binary),由于并没有数据拷贝所以binary的模式匹配成本相当低
match context类似sub binary,但是针对二进制匹配做了优化;例如它包含一个直接指向二进制数据的指针.从二进制匹配出来字段值之后移动指针位置即可.

官方文档链接:http://www.erlang.org/doc/efficiency_guide/binaryhandling.html

Internally, binaries and bitstrings are implemented in the same way.

There are four types of binary objects internally. Two of them are containers for binary data and two of them are merely references to a part of a binary.

The binary containers are called refc binaries (short for reference-counted binaries) and heap binaries.

Refc binaries consist of two parts: an object stored on the process heap, called a ProcBin, and the binary object itself stored outside all process heaps.The binary object can be referenced by any number of ProcBins from any number of processes; the object contains a reference counter to keep track of the number of references, so that it can be removed when the last reference disappears.

All ProcBin objects in a process are part of a linked list, so that the garbage collector can keep track of them and decrement the reference counters in the binary when a ProcBin disappears.

Heap binaries are small binaries, up to 64 bytes, that are stored directly on the process heap. They will be copied when the process is garbage collected and when they are sent as a message. They don't require any special handling by the garbage collector.

There are two types of reference objects that can reference part of a refc binary or heap binary. They are called sub binaries and match contexts.

A sub binary is created by split_binary/2 and when a binary is matched out in a binary pattern. A sub binary is a reference into a part of another binary (refc or heap binary, never into a another sub binary). Therefore, matching out a binary is relatively cheap because the actual binary data is never copied.
A match context is similar to a sub binary, but is optimized for binary matching; for instance, it contains a direct pointer to the binary data. For each field that is matched out of a binary, the position in the match context will be incremented.

本文链接

[Erlang 0031] Erlang Shell中的输出完整数据

2012-01-10T14:28:00Z

前两天群里面有人问shell里面长数据被省略为 [...]|...],如何查看被省略的部分,他是在调用os:getenv()的时候遇到的这个问题,咱们前面也遇到过类似的问题,比如 string:tokens(binary_to_list(erlang:system_info(info)),"\n").

Eshell V5.9  (abort with ^G)
1>  string:tokens( binary_to_list(erlang:system_info(info)),"\n").
["=memory","total: 4331920","processes: 438877",
"processes_used: 438862","system: 3893043","atom: 146321",
"atom_used: 119102","binary: 327936","code: 1929551",
"ets: 123308","=hash_table:atom_tab","size: 4813",
"used: 3508","objs: 6410","depth: 7",
"=index_table:atom_tab","size: 7168","limit: 1048576",
"entries: 6410","=hash_table:module_code","size: 97",
"used: 52","objs: 72","depth: 4","=index_table:module_code",
"size: 1024","limit: 65536","entries: 72",
[...]|...]

当时我在群里面给了一个不标准的解决方法:binary_to_list(list_to_binary(os:getenv())).这个很野的路子是以前调试代码的时候偶尔发现的;

标准的解决方案是使用shell中的rp方法格式化输出结果比如 rp(os:getenv()).特此修正,抱歉

9> L=lists:seq(1,100).
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,
23,24,25,26,27,28,29|...]
10> rp(L).
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,
23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,
43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,
63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,
83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100]
ok
11>

有必要了解一下shell中提供的方法:

Eshell V5.9  (abort with ^G)
1> help().
** shell internal commands **
b()        -- display all variable bindings
e(N)       -- repeat the expression in query <N>
f()        -- forget all variable bindings
f(X)       -- forget the binding of variable X
h()        -- history
history(N) -- set how many previous commands to keep
results(N) -- set how many previous command results to keep
catch_exception(B) -- how exceptions are handled
v(N)       -- use the value of query <N>
rd(R,D)    -- define a record
rf()       -- remove all record information
rf(R)      -- remove record information about R
rl()       -- display all record information
rl(R)      -- display record information about R
rp(Term)   -- display Term using the shell's record information
rr(File)   -- read record information from File (wildcards allowed)
rr(F,R)    -- read selected record information from file(s)
rr(F,R,O)  -- read selected record information with options
** commands in module c **
bt(Pid)    -- stack backtrace for a process
c(File)    -- compile and load code in <File>
cd(Dir)    -- change working directory
flush()    -- flush any messages sent to the shell
help()     -- help info
i()        -- information about the system
ni()       -- information about the networked system
i(X,Y,Z)   -- information about pid <X,Y,Z>
l(Module)  -- load or reload module
lc([File]) -- compile a list of Erlang modules
ls()       -- list files in the current directory
ls(Dir)    -- list files in directory <Dir>
m()        -- which modules are loaded
m(Mod)     -- information about module <Mod>
memory()   -- memory allocation information
memory(T)  -- memory allocation information of type <T>
nc(File)   -- compile and load code in <File> on all nodes
nl(Module) -- load module on all nodes
pid(X,Y,Z) -- convert X,Y,Z to a Pid
pwd()      -- print working directory
q()        -- quit - shorthand for init:stop()
regs()     -- information about registered processes
nregs()    -- information about all registered processes
xm(M)      -- cross reference check a module
y(File)    -- generate a Yecc parser
** commands in module i (interpreter interface) **
ih()       -- print help for the i module
true
2>

这些方法的内部实现为:

bi(I)           -> c:bi(I).
bt(Pid)          -> c:bt(Pid).
c(File)      -> c:c(File).
c(File, Opt)    -> c:c(File, Opt).
cd(D)           -> c:cd(D).
erlangrc(X)      -> c:erlangrc(X).
flush()         -> c:flush().
i()           -> c:i().
i(X,Y,Z)      -> c:i(X,Y,Z).
l(Mod)            -> c:l(Mod).
lc(X)            -> c:lc(X).
ls()            -> c:ls().
ls(S)           -> c:ls(S).
m()           -> c:m().
m(Mod)           -> c:m(Mod).
memory()        -> c:memory().
memory(Type)    -> c:memory(Type).
nc(X)          -> c:nc(X).
ni()            -> c:ni().
nl(Mod)      -> c:nl(Mod).
nregs()         -> c:nregs().
pid(X,Y,Z)      -> c:pid(X,Y,Z).
pwd()           -> c:pwd().
q()          -> c:q().
regs()          -> c:regs().
xm(Mod)         -> c:xm(Mod).
y(File)         -> c:y(File).
y(File, Opts)   -> c:y(File, Opts).

代码参见:http://www.google.com/codesearch#dNzC48WMK6w/lib/stdlib/src/shell_default.erl&ct=rc&cd=1&q=shell_default%20lang:erlang

晚安!

本文链接

[Erlang 0030] 理解Erlang/OTP Supervisor

2012-01-10T06:17:00Z

Supervisors are used to build an hierarchical process structure called a supervision tree, a nice way to structure a fault tolerant application.
--Erlang/OTP Doc

Supervisor的基本思想就是通过建立层级结构实现错误隔离和管理,具体方法是通过重启的方式保持子进程一直活着.如果supervisor是进程树的一部分,它会被它的supervisor自动终止,当它的supervisor让它shutdown的时候,它会按照子进程启动顺序的逆序终止其所有的子进程,最后终止掉自己.[Erlang 0025]理解Erlang/OTP - Application以log4erl项目为学习了Erlang/OTP application,我们说到application在start的方法中启动了log4erl的顶层监控树.今天我们继续跟进,看log4erl的监控树是怎么构建起来的,并做实验看supervisor如何通过重启恢复服务的.使用application:start(log4erl).启动起来之后的进程树:

下面是log4erl_sup文件的start_link方法,supervisor:start_link方法的执行是同步的,直到所有的子进程都启动了才会返回. supervisor:start_link会使用回调函数init/1.

start_link(Default_logger) ->
    R = supervisor:start_link({local, ?MODULE}, ?MODULE, []),
    %log4erl:start_link(Default_logger),
    add_logger(Default_logger),
    ?LOG2("Result in supervisor is ~p~n",[R]),
    R.

%%回调的方法init/1
init([]) ->
    {ok,
     {
       {one_for_one,3,10}, 
       []
      }
    }.

log4erl的顶层监控树的初始化相当简单仅仅定义了重启策略(RestartStrategy)和最大重启频率(maximum restart frequency):{one_for_one,3,10}.

{one_for_one,3,10}表达的语义是{How, Max, Within}:在多长时间内(Within)重启了几次(Max),如何重启(HOW 重启策略);设计最大重启频率是为了避免反复重启进入死循环,一旦超出了此阈值,supervisor进程会结束掉自己以及它所有的子进程,并通过进程树传递退出消息,更上层的supervisor就会采取适当的措施,要么重启终止的supervisor要么自己也终止掉.可能比较纠结这几个值怎么配置,多数资料上都会告诉你"如何配置完全取决于你的应用程序".这个还是有经验值的,生成环境的经验值是一小时内重启4次,也可以参考一些和你应用类似的开源项目看看它们是怎么配置的.如果填写的是{one_for_one,0,1}就是不允许重启,下面的示例中可以看到YAWS项目采用了这样的策略.

下面几个开源项目顶层supervisor的init方法:

%%rabbit_sup.erl  来自大名鼎鼎的rabbitmq
init([]) ->
    {ok, {{one_for_all, 0, 1}, []}}.


%%yaws_sup.erl  Yaws项目 - Yet Another Web Server
init([]) ->

    ChildSpecs = child_specs(),
    %% 0, 1 means that we never want supervisor restarts
    {ok,{{one_for_all, 0, 1}, ChildSpecs}}.


%%ejabberd_sup ejabberd项目
init([]) ->
    Hooks =
     {ejabberd_hooks,
     {ejabberd_hooks, start_link, []},
   %%......................... 省略代码
    {ok, {{one_for_one, 10, 1},
       [Hooks,
           GlobalRouter,
        Cluster,
      ..................
        Listener]}}.

重启策略

one-for-one : 把子进程当成各自独立的,一个进程出现问题其它进程不会受到崩溃的进程的影响.该子进程死掉,只有这个进程会被重启
one_for_all : 如果子进程终止,所有其它子进程也都会被终止,然后所有进程都会被重启.
rest_for_one:如果一个子进程终止,在这个进程启动之后启动的进程都会被终止掉.然后终止掉的进程和连带关闭的进程都会被重启.
simple_one_for_one 是one_for_one的简化版 ,所有子进程都动态添加同一种进程的实例

在log4erl_sup.erl的start_link中启动了顶层supervisor之后,添加了一个默认的logger: add_logger(Default_logger),

add_logger(Name) when is_atom(Name) ->
    N = atom_to_list(Name),
    add_logger(N);
add_logger(Name) when is_list(Name) ->
    C1 = {Name,
       {log_manager, start_link ,[Name]},
       permanent,
       10000,
       worker,
       [log_manager]},
   
    ?LOG2("Adding ~p to ~p~n",[C1, ?MODULE]),
    supervisor:start_child(?MODULE, C1).

添加的logger是log4erl_sup的子进程,子进程启动和监控的方式通过child specification来指定.

C1 = {Name,{log_manager, start_link ,[Name]},permanent,10000,worker,[log_manager]}

C1的六个数据项分别为: {ID, StartEntery, Restart, Shutdown, Type, Modules}:

ID :supervisor 用来在内部区分specification的,所以只要子进程规格说明之间不重复就可以.

Start : 启动参数{M,F,A}
Restart : 这个进程遇到错误之后是否重启
permanent:遇到任何错误导致进程终止就会重启
temporary:进程永远都不会被重启
transient: 只有进程异常终止的时候会被重启
Shutdown 进程如何被干掉,这里是使用整型值2000的意思是,进程在被强制干掉之前有2000毫秒的时间料理后事自行终止.
实际过程是supervisor给子进程发送一个exit(Pid,shutdown)然后等待exit信号返回,在指定时间没有返回则将子进程使用exit(Child,kill)
这里的参数还有 brutal_kill 意思是进程马上就会被干掉
infinity :当一个子进程是supervisor那么就要用infinity,意思是给supervisor足够的时间进行重启.
Type 这里只有两个值:supervisor worker ; 只要没有实现supervisor behavior的进程都是worker;

可以通过supervisor的层级结构来精细化对进程的控制.这个值主要作用是告知监控进程它的子进程是supervisor还是worker
Modules 是进程依赖的模块,这个信息只有在代码热更新的时候才会被用到:标注了哪些模块需要按照什么顺序进行更新;通常这里只需要列出进程依赖的主模块. 如果子进程是supervisor gen_server gen_fsm Module名是回调模块的名称,这时Modules的值是只有一个元素的列表,元素就是回调模块的名称;如果子进程是gen_event Modules的值是 dynamic;关于dynamic参数余锋有一篇专门的分析:Erlang supervisor规格的dynamic行为分析 http://blog.yufeng.info/archives/1455

实际应用中log4erl中的logger会根据业务逻辑添加多个,我们也不是直接通过application:start(log4erl).而是调用 log4erl:conf(log4erl.conf)这个方法简单的封装了内部逻辑,实际调用的是 log4erl_conf:conf(File).我们这里定义一个简单的log4erl.conf文件,使用log4erl:conf(log4erl.conf).启动之后,我们看看它的进程树是什么样的:

%%log4erl.conf文件内容我做了简单的缩排

%%mod
logger default_logger{
     file_appender default_app{
    dir = "./log", level = debug, file = default_log, type = size, max = 1000000, suffix = log, rotation = 50, format = ' %d %h:%m:%s.%i %l%n'
     }
}

%%mail mod
logger mail_logger{
     file_appender mail_app{
     dir = "./log", level = debug, file = mail_log, type = size, max = 1000000, suffix = log, rotation = 50, format = ' %d %h:%m:%s.%i %l%n'
     }
}

对应的进程树是这样的,进程之间的红线表示link关系:

我们沿着调用关系,逐步跟进代码:

%==== File : log4erl_conf =======

%%log4erl_conf:conf(File).
conf(File) ->
    application:start(log4erl),     %%启动log4erl
    Tree = parse(leex(File)),     %%解析配置文件
    traverse(Tree).                   %%遍历配置项构造监控树

%%跟进遍历的逻辑,对于每一条配置执行的是element/1方法
traverse([]) ->
    ok;
traverse([H|Tree]) ->
    element(H),
    traverse(Tree).

%%对于我们自定义的logger走的是{logger, Logger, Appenders}逻辑
element({cutoff_level, CutoffLevel}) ->
    log_filter_codegen:set_cutoff_level(CutoffLevel);
element({default_logger, Appenders}) ->
    appenders(Appenders);
element({logger, Logger, Appenders}) ->
    log4erl:add_logger(Logger),                
    appenders(Logger, Appenders). 

%==== File : log4erl  =======
%%继续跟进我们走到log4erl:add_logger/1
add_logger(Logger) ->
    try_msg({add_logger, Logger}).

%%try_msg 是的添加了异常捕获的通用方法
try_msg(Msg) ->
     try
     handle_call(Msg)
     catch
     exit:{noproc, _M} ->
          io:format("log4erl has not been initialized yet. To do so, please run~n"),
          io:format("> application:start(log4erl).~n"),
          {error, log4erl_not_started};
        E:M ->
          ?LOG2("Error message received by log4erl is ~p:~p~n",[E, M]),
          {E, M}
     end.

%%handle_call的代码片段
handle_call({add_logger, Logger}) ->
    log_manager:add_logger(Logger);

%==== File :  log_manager  =======
%%逻辑转到log_manager的add_logger(Logger)
%%最终调用的是log4erl_sup:add_logger(Logger).这个我们上面已经分析过了
add_logger(Logger) ->
    log4erl_sup:add_logger(Logger).

%%element方法在添加loger之后会添加appender
appenders([]) ->
    ok;
appenders([H|Apps]) ->
    appender(H),
    appenders(Apps).

appenders(_, []) ->
    ok;
appenders(Logger, [H|Apps]) ->
    appender(Logger, H),
    appenders(Logger, Apps).

appender({appender, App, Name, Conf}) ->
    log4erl:add_appender({App, Name}, {conf, Conf}).

appender(Logger, {appender, App, Name, Conf}) ->
    log4erl:add_appender(Logger, {App, Name}, {conf, Conf}).


%==== File :  log4erl  =======
%% Appender = {Appender, Name}
add_appender(Logger, Appender, Conf) ->
    try_msg({add_appender, Logger, Appender, Conf}).

handle_call({add_appender, Logger, Appender, Conf}) ->
    log_manager:add_appender(Logger, Appender, Conf);

%==== File :  log_manager =======
add_appender(Logger, {Appender, Name} , Conf) ->
    ?LOG2("add_appender ~p with name ~p to ~p with Conf ~p ~n",[Appender, Name, Logger, Conf]),
    log4erl_sup:add_guard(Logger, Appender, Name, Conf).

%==== File :  log4erl_sup =======
add_guard(Logger, Appender, Name, Conf) ->
    C = {Name,
     {logger_guard, start_link ,[Logger, Appender, Name, Conf]},
     permanent,
     10000,
     worker,
     [logger_guard]},
    ?LOG2("Adding ~p to ~p~n",[C, ?MODULE]),
    supervisor:start_child(?MODULE, C).

%==== File :  logger_guard =======
start_link(Logger, Appender, Name, Conf) ->
    %?LOG2("starting guard for logger ~p~n",[Logger]),
    {ok, Pid} = gen_server:start_link(?MODULE, [Appender, Name], []),
    case add_sup_handler(Pid, Logger, Conf) of
     {error, E} ->
         gen_server:call(Pid, stop),
         {error, E};
     _R ->
         {ok, Pid}
    end.

add_sup_handler(G_pid, Logger, Conf) ->
    ?LOG("add_sup()~n"),
    gen_server:call(G_pid, {add_sup_handler, Logger, Conf}).

handle_call({add_sup_handler, Logger, Conf}, _From, [{appender, Appender, Name}] = State) ->
    ?LOG2("Adding handler ~p with name ~p for ~p From ~p~n",[Appender, Name, Logger, _From]),
    try
     Res = gen_event:add_sup_handler(Logger, {Appender, Name}, Conf),
     {reply, Res, State}
    catch
     E:R ->
         {reply, {error, {E,R}}, State}
    end;

gen_event:add_sup_handler会建立EventManager与Event Handler之间的link的关系,所以我们修改一下,注释掉这段,看看监控树是什么样子:

add_sup_handler(G_pid, Logger, Conf) ->

% ?LOG("add_sup()~n"),
% gen_server:call(G_pid, {add_sup_handler, Logger, Conf}).
ok.

注释掉之后可以看到logger和guard之间的link关系就不存在了.

kill进程的实验

后面我们会用各种情况杀掉进程,看这个进程树对异常的处理情况;我们的实验步骤:

1.发送退出消息Reason:some_reason给default_logger

2.发送退出消息Reason:kill 给default_logger

3.发送退出消息Reason:some_reason给logger_guard

4.发送退出消息Reason:some_reason给log4erl_sup

5.发送退出消息Reason:kill 给log4erl_sup

3> whereis(default_logger).
<0.45.0>
4> exit(whereis(default_logger),some_reason).
true
5> whereis(default_logger).
<0.45.0>
6> exit(whereis(default_logger),some_reason). %%由于gen_event默认process_flag(trap_exit, true),所以some_reason的退出消息并没有把它干掉
true
7> whereis(default_logger).
<0.45.0>
8> exit(whereis(default_logger),kill).    %%向进程发送强制退出消息,
true

=SUPERVISOR REPORT==== 10-Jan-2012::10:35:21 ===      %首先能够看到log4erl报出的子进程终止的报告
     Supervisor: {local,log4erl_sup}
     Context:    child_terminated
     Reason:     killed
     Offender:   [{pid,<0.45.0>},
                  {name,"default_logger"},
                  {mfargs,{log_manager,start_link,["default_logger"]}},
                  {restart_type,permanent},
                  {shutdown,10000},
                  {child_type,worker}]

=PROGRESS REPORT==== 10-Jan-2012::10:35:21 ===      %log4erl_sup重建default_logger,新进程pid是<0.69.0>
          supervisor: {local,log4erl_sup}
             started: [{pid,<0.69.0>},
                       {name,"default_logger"},
                       {mfargs,{log_manager,start_link,["default_logger"]}},
                       {restart_type,permanent},
                       {shutdown,10000},
                       {child_type,worker}]

=SUPERVISOR REPORT==== 10-Jan-2012::10:35:21 ===  %default_logger退出消息转变成为killed继续广播给link的进程,对应的logger_guard终止
     Supervisor: {local,log4erl_sup}
     Context:    child_terminated
     Reason:     killed
     Offender:   [{pid,<0.46.0>},
                  {name,default_app},
                  {mfargs,
                      {logger_guard,start_link,
                          [default_logger,file_appender,default_app,
                           {conf, [{dir,"./log"},{level,debug},{file,default_log},{type,size},
                                {max,1000000},{suffix,log}, {rotation,50},
                                {format," %d %h:%m:%s.%i %l%n"}]}]}},
                  {restart_type,permanent},
                  {shutdown,10000},
                  {child_type,worker}]

=PROGRESS REPORT==== 10-Jan-2012::10:35:21 === %logger_guard 重建
          supervisor: {local,log4erl_sup}
             started: [{pid,<0.70.0>},
                       {name,default_app},
                       {mfargs,
                           {logger_guard,start_link,
                               [default_logger,file_appender,default_app,
                                {conf,
                                    [{dir,"./log"},{level,debug}, {file,default_log},{type,size},
                                     {max,1000000}, {suffix,log},{rotation,50},
                                     {format," %d %h:%m:%s.%i %l%n"}]}]}},
                       {restart_type,permanent},
                       {shutdown,10000},
                       {child_type,worker}]
9> whereis(default_logger).       
<0.69.0>
10> is_process_alive(pid(0,70,0)).   %这是新启动的logger_guard进程
true
11> exit(pid(0,70,0),some_reason).   %向进程发送一个退出消息
true

=SUPERVISOR REPORT==== 10-Jan-2012::11:07:51 ===
     Supervisor: {local,log4erl_sup}
     Context:    child_terminated
     Reason:     some_reason
     Offender:   [{pid,<0.70.0>},
                  {name,default_app},
                  {mfargs,
                      {logger_guard,start_link,
                          [default_logger,file_appender,default_app,
                           {conf,
                               [{dir,"./log"},{level,debug},{file,default_log},{type,size},{max,1000000},
                                {suffix,log},{rotation,50},{format," %d %h:%m:%s.%i %l%n"}]}]}},
                  {restart_type,permanent},
                  {shutdown,10000},
                  {child_type,worker}]

12>
=PROGRESS REPORT==== 10-Jan-2012::11:07:51 ===
          supervisor: {local,log4erl_sup}
             started: [{pid,<0.76.0>},
                       {name,default_app},
                       {mfargs,
                           {logger_guard,start_link,
                               [default_logger,file_appender,default_app,
                                {conf,
                                    [{dir,"./log"},{level,debug},{file,default_log},{type,size},{max,1000000},
                                     {suffix,log},{rotation,50},{format," %d %h:%m:%s.%i %l%n"}]}]}},
                       {restart_type,permanent},
            {shutdown,10000},{child_type,worker}]
12> is_process_alive(pid(0,70,0)).
false
13> whereis(default_logger).  %退出消息广播对default_logger没有影响
<0.69.0>
14> whereis(log4erl_sup).
<0.44.0>
15> exit(whereis(log4erl_sup),some_reason).  % Supervisor 初始化的时候也会设置  process_flag(trap_exit, true),
true
16> whereis(log4erl_sup).
<0.44.0>
17> exit(whereis(log4erl_sup),kill).  %杀掉log4erl_sup 应用程序停止
true

=CRASH REPORT==== 10-Jan-2012::13:26:23 ===
  crasher:
    initial call: gen_event:init_it/6
    pid: <0.69.0>
    registered_name: default_logger
    exception exit: killed
      in function  gen_event:terminate_server/4
    ancestors: [log4erl_sup,<0.43.0>]
    messages: [{'EXIT',<0.76.0>,killed}]
    links: [#Port<0.1891>,#Port<0.1885>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 24
    reductions: 720
  neighbours:
18>
=CRASH REPORT==== 10-Jan-2012::13:26:23 ===
  crasher:
    initial call: gen_event:init_it/6
    pid: <0.47.0>
    registered_name: mail_logger
    exception exit: killed
      in function  gen_event:terminate_server/4
    ancestors: [log4erl_sup,<0.43.0>]
    messages: [{'EXIT',<0.48.0>,killed}]
    links: [#Port<0.546>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 377
    stack_size: 24
    reductions: 411
  neighbours:
18>
=CRASH REPORT==== 10-Jan-2012::13:26:25 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.42.0>
    registered_name: []
    exception exit: killed
      in function  application_master:terminate/2
    ancestors: [<0.41.0>]
    messages: []
    links: [<0.6.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 24
    reductions: 1555
  neighbours:
18>
=INFO REPORT==== 10-Jan-2012::13:26:25 ===
    application: log4erl
    exited: killed
    type: temporary
18>

最后再贴一次log4erl项目的地址: http://code.google.com/p/log4erl/,建议下载下来代码自己动手做一下上面的实验.

下一篇:[Erlang 0031] Erlang Shell中的输出完整数据

本文链接

[Erlang 0029] Erlang Inline编译

2012-01-09T05:40:00Z

内联函数建议编译器将制定的函数体插入并取代每一处调用该函数的地方,从而节省了每一次函数调用带来的时间开支,选择内联函数时,必须要在程序占用空间和程序执行效率之间进行权衡,因为过多的对较为复杂的函数进行内联扩展将带来很大的存储资源开支.另外注意对于递归函数的内联扩展可能带来部分编译器的无穷编译.内联扩展是一种特别的用于消除调用函数时所造成的固有的时间消耗方法。一般用于能够快速执行的函数，因为在这种情况下函数调用的时间消耗显得更为突出。
--维基百科内联函数摘要

从维基百科的描述中可以看到内联函数解决的问题是:函数的调用时间比函数执行时间相当的时候,通过空间换时间,获得执行效率.实现角度Inlining是通过代码复制的方式节省进栈出栈的开销.Erlang的编译器可以将Erlang模块中的函数进行内联编译,内联(inlining)的含义是把一个方法的调用替换成函数体并把参数替换成实际值.
Erlang内联不是默认值;必须明确指定compile选项( 形式: {inline,[{Name,Arity},...]} ) 或者在源代码使用-compile.

%%Example of explicit inlining:
-compile({inline,[pi/0]}).
pi() -> 3.1416.

%% Example of implicit inlining:
-compile(inline).

%% Aggressive inlining - will increase code size.
-compile(inline).
-compile({inline_size,100}).

如果一个函数被编译成inline,原始的函数还是会被保留,我们可以直接在erlang shell中调用这个方法.

-module(test).
-compile(export_all).
-compile({inline,[server_id/0]}).

server_id() ->
               2396.

内联编译不一定提高运行时的效率.例如内联可能增加Beam的栈消耗,对于递归函数调用这显然是有损性能的.{inline_size,Size}就是用来控制方法在多大程度上可以inline.Size默认值是24,这样inline代码与没有做inline的代码size相当,只有相当小的方法会被做inline.

那这个Size到底是指什么的大小呢?是代码函数?是代码个数?还是别的什么?我在erlangqa.org提了这个问题,得到了litaocheng的解答:

http://www.erlangqa.com/?qa=100/inline_size-size-%E4%B8%AD%E7%9A%84size%E6%98%AF%E6%8C%87%E4%BB%80%E4%B9%88

请参考otp_src/compiler/src/cerl_inline.erl的weight/1函数。
相应的Erlang表达式都有不同的权重。inline_size指的是函数汇编后的权重值。
可以通过 erlc +\'S\' your.erl来得到汇编文件:your.S。

同时参考cerl_inline.erl文件中：当inline_size为30时，90%的情况下可以得到最大加速。inline_size为100-150时，98%的情况下可以最大优化。如果指定更大值，会使代码尺寸变大，性能反而受到影响。

按图索骥找到cerl_inline.erl weight/1的代码:

 1 default_effort() -> 150.
 2 default_size() -> 24.
 3 default_unroll() -> 1.
 4 
 5 %% Base costs/weights for different kinds of expressions. If these are
 6 %% modified, the size limits above may have to be adjusted.
 7 
 8 weight(var) -> 0;     % We count no cost for variable accesses.
 9 weight(values) -> 0;     % Value aggregates have no cost in themselves.
10 weight(literal) -> 1;     % We assume efficient handling of constants.
11 weight(data) -> 1;     % Base cost; add 1 per element.
12 weight(element) -> 1;   % Cost of storing/fetching an element.
13 weight(argument) -> 1;  % Cost of passing a function argument.
14 weight('fun') -> 6;     % Base cost + average number of free vars.
15 weight('let') -> 0;     % Count no cost for let-bindings.
16 weight(letrec) -> 0;    % Like a let-binding.
17 weight('case') -> 0;     % Case switches have no base cost.
18 weight(clause) -> 1;    % Count one jump at the end of each clause body.
19 weight('receive') -> 9;     % Initialization/cleanup cost.
20 weight('try') -> 1;     % Assume efficient implementation.
21 weight('catch') -> 1;     % See `try'.
22 weight(apply) -> 3;     % Average base cost: call/return.
23 weight(call) -> 3;      % Assume remote-calls as efficient as `apply'.
24 weight(primop) -> 2;    % Assume more efficient than `apply'.
25 weight(binary) -> 4;    % Initialisation base cost.
26 weight(bitstr) -> 3;    % Coding/decoding a value; like a primop.
27 weight(module) -> 1.    % Like a letrec with a constant body

也就是说,不同的表达式有不同的权重值,Size既不是代码函数也不是函数个数,而是依赖于该权重值.这里我不再继续跟进了,cerl_inline.erl的注释提供了详细的信息:

Normal execution times for inlining are between 0.1 and 0.3 seconds (on the author's current equipment). The default effort limit of 150 is high enough that most normal programs never hit the limit even once, and for difficult programs, it generally keeps the execution times below 2-5 seconds. Using an effort counter of 1000 will thus have no further effect on most programs, but some programs may take as much as 10 seconds or more. Effort counts larger than 2500 have never been observed even on very ill-conditioned programs.

Size limits between 6 and 18 tend to actually shrink the code, because of the simplifications made possible by inlining. A limit of 16 seems to be optimal for this purpose, often shrinking the executable code by up to 10%. Size limits between 18 and 30 generally give the same code size as if no inlining was done (i.e., code duplication balances out the simplifications at these levels). A size limit between 1 and 5 tends to inline small functions and propagate constants, but does not cause much simplifications do be done, so the net effect will be a slight increase in code size. For size limits above 30, the executable code size tends to increase with about 10% per 100 units, with some variations depending on the sizes of functions in the source code.

Typically, about 90% of the maximum speedup achievable is already reached using a size limit of 30, and 98% is reached at limits around 100-150; there is rarely any point in letting the code size increase by more than 10-15%. If too large functions are inlined, cache effects will slow the program down.

感兴趣的可以找到原始论文看下,论文地址: "Fast and Effective Procedure Inlining", International Static Analysis Symposium 1997 http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.2438

算法不跟进,但是实验要做的,就按照litaocheng建议的方法,我做了一个这样的demo,写一个简单的方法,这个方法在另一个方法里面被多次调用(代码如下).我是在windows环境中使用,在shell中使用命令c(test,['S']).得到assembler code文件.

-module(test).
-export([get_name/0, show/1]).

get_name() ->
     "This is Test Module".

show(A) ->
     A=get_name(),
     A=get_name(),
     A=get_name(),
     A=get_name(),
     A=get_name(),
     A=get_name(),
     io:format("This is ~p ~n",[A]).

生成的assembler code文件如下:

{module, test}.  %% version = 0

{exports, [{get_name,0},{module_info,0},{module_info,1},{show,1}]}.

{attributes, []}.

{labels, 15}.


{function, get_name, 0, 2}.
  {label,1}.
    {line,[{location,"test.erl",22}]}.
    {func_info,{atom,test},{atom,get_name},0}.
  {label,2}.
    {move,{literal,"This is Test Module"},{x,0}}.
    return.


{function, show, 1, 4}.
  {label,3}.
    {line,[{location,"test.erl",25}]}.
    {func_info,{atom,test},{atom,show},1}.
  {label,4}.
    {allocate,1,1}.
    {move,{x,0},{y,0}}.
    {line,[{location,"test.erl",26}]}.
    {call,0,{f,2}}.
    {test,is_eq_exact,{f,5},[{x,0},{y,0}]}.
    {line,[{location,"test.erl",27}]}.
    {call,0,{f,2}}.
    {test,is_eq_exact,{f,6},[{x,0},{y,0}]}.
    {line,[{location,"test.erl",28}]}.
    {call,0,{f,2}}.
    {test,is_eq_exact,{f,7},[{x,0},{y,0}]}.
    {line,[{location,"test.erl",29}]}.
    {call,0,{f,2}}.
    {test,is_eq_exact,{f,8},[{x,0},{y,0}]}.
    {line,[{location,"test.erl",30}]}.
    {call,0,{f,2}}.
    {test,is_eq_exact,{f,9},[{x,0},{y,0}]}.
    {line,[{location,"test.erl",31}]}.
    {call,0,{f,2}}.
    {test,is_eq_exact,{f,10},[{x,0},{y,0}]}.
    {test_heap,2,0}.
    {put_list,{y,0},nil,{x,1}}.
    {move,{literal,"This is ~p ~n"},{x,0}}.
    {line,[{location,"test.erl",32}]}.
    {call_ext_last,2,{extfunc,io,format,2},1}.
  {label,5}.
    {line,[{location,"test.erl",26}]}.
    {badmatch,{x,0}}.
  {label,6}.
    {line,[{location,"test.erl",27}]}.
    {badmatch,{x,0}}.
  {label,7}.
    {line,[{location,"test.erl",28}]}.
    {badmatch,{x,0}}.
  {label,8}.
    {line,[{location,"test.erl",29}]}.
    {badmatch,{x,0}}.
  {label,9}.
    {line,[{location,"test.erl",30}]}.
    {badmatch,{x,0}}.
  {label,10}.
    {line,[{location,"test.erl",31}]}.
    {badmatch,{x,0}}.


{function, module_info, 0, 12}.
  {label,11}.
    {line,[]}.
    {func_info,{atom,test},{atom,module_info},0}.
  {label,12}.
    {move,{atom,test},{x,0}}.
    {line,[]}.
    {call_ext_only,1,{extfunc,erlang,get_module_info,1}}.


{function, module_info, 1, 14}.
  {label,13}.
    {line,[]}.
    {func_info,{atom,test},{atom,module_info},1}.
  {label,14}.
    {move,{x,0},{x,1}}.
    {move,{atom,test},{x,0}}.
    {line,[]}.
    {call_ext_only,2,{extfunc,erlang,get_module_info,2}}.

添加了inline选项之后的结果,这个demo比较变态生成的代码效果也比较明显:

{module, test}.  %% version = 0

{exports, [{get_name,0},{module_info,0},{module_info,1},{show,1}]}.

{attributes, []}.

{labels, 10}.


{function, get_name, 0, 2}.
  {label,1}.
    {line,[{location,"test.erl",23}]}.
    {func_info,{atom,test},{atom,get_name},0}.
  {label,2}.
    {move,{literal,"This is Test Module"},{x,0}}.
    return.


{function, show, 1, 4}.
  {label,3}.
    {line,[{location,"test.erl",26}]}.
    {func_info,{atom,test},{atom,show},1}.
  {label,4}.
    {test,is_eq_exact,{f,5},[{literal,"This is Test Module"},{x,0}]}.
    {test_heap,2,1}.
    {put_list,{x,0},nil,{x,1}}.
    {move,{literal,"This is ~p ~n"},{x,0}}.
    {line,[{location,"test.erl",33}]}.
    {call_ext_only,2,{extfunc,io,format,2}}.
  {label,5}.
    {line,[{location,"test.erl",27}]}.
    {badmatch,{literal,"This is Test Module"}}.


{function, module_info, 0, 7}.
  {label,6}.
    {line,[]}.
    {func_info,{atom,test},{atom,module_info},0}.
  {label,7}.
    {move,{atom,test},{x,0}}.
    {line,[]}.
    {call_ext_only,1,{extfunc,erlang,get_module_info,1}}.


{function, module_info, 1, 9}.
  {label,8}.
    {line,[]}.
    {func_info,{atom,test},{atom,module_info},1}.
  {label,9}.
    {move,{x,0},{x,1}}.
    {move,{atom,test},{x,0}}.
    {line,[]}.
    {call_ext_only,2,{extfunc,erlang,get_module_info,2}}.

官方文档: http://www.erlang.org/doc/man/compile.html

Erlang Assembly Code处于 not documented的状态,下面有两篇相关的文章:

Howto dump Core Erlang and assembly http://untroubled.be/docs/erlang/howto_dump_core_erlang_and_assembly.html

How to modify Erlang assembly? Are any resources available? http://stackoverflow.com/questions/7935054/how-to-modify-erlang-assembly-are-any-resources-available

本文链接

[Erlang 0028] Erlang atom

2012-01-04T07:57:00Z

Erlang中atom数据类型能够做的唯一的运算就是比较;在erlang中模块名和方法名都是原子; Erlang M-F-A方法调用可以做的非常灵活,我们在shell里面操练一下:

Eshell V5.9  (abort with ^G)
1> lists:seq(1,5).
[1,2,3,4,5]
2> L=lists.
lists
3> S=seq.
seq
4> L:S(1,5).
[1,2,3,4,5]
5> L:seq(1,5).
[1,2,3,4,5]
6> L2=list_to_atom("list" ++"s").
lists
7> L2:seq(1,5).
[1,2,3,4,5]
8> apply(list_to_atom("li"++"sts"),seq,[1,5]).
[1,2,3,4,5]
9>

Erlang atom不参与垃圾回收,一旦创建就不会被移除掉;一旦超出atom的数量限制(默认是1048576) VM就会终止掉.对于一个会持续运行很久的系统,把任意字符串转成atom是很危险的,内存会慢慢被吃光.如果使用的原子是在预期范围内的,比如协议模块的名称,那么可以使用list_to_existing_atom来进行防范,这个方法所产出的atom必须是之前已经创建过的.

%% list_to_existing_atom demo
 
Eshell V5.9  (abort with ^G)
1> list_to_existing_atom("player_1").
** exception error: bad argument              %尝试调用 list_to_existing_atom("player_1").由于player_1的原子之前没有被创建过
     in function  list_to_existing_atom/1        %这里报错了 异常是bad argument
        called as list_to_existing_atom("player_1")
2> list_to_atom("player_1").                      %创建一下player_1
player_1
3> list_to_existing_atom("player_1").          %再次调用list_to_existing_atom就是对的了
player_1
4> list_to_existing_atom("player_2").          
** exception error: bad argument
     in function  list_to_existing_atom/1
        called as list_to_existing_atom("player_2")
5> player_2.                                             %也可以这样创建原子 
player_2
6> list_to_existing_atom("player_2").          %这时调用也是对的
player_2
7>

我们可以使用string:tokens( binary_to_list(erlang:system_info(info)),"\n")在shell中看一下atom的使用情况,输出的片段中恰好包含了原子内存使用的情况,当前数量和数量限制;想看完整的输出?穿越到这里

%% list_to_atom demo limit

Eshell V5.9  (abort with ^G)
1>  string:tokens( binary_to_list(erlang:system_info(info)),"\n").
["=memory","total: 4331920","processes: 438877",
"processes_used: 438862","system: 3893043","atom: 146321",
"atom_used: 119102","binary: 327936","code: 1929551",
"ets: 123308","=hash_table:atom_tab","size: 4813",
"used: 3508","objs: 6410","depth: 7",
"=index_table:atom_tab","size: 7168","limit: 1048576",
"entries: 6410","=hash_table:module_code","size: 97",
"used: 52","objs: 72","depth: 4","=index_table:module_code",
"size: 1024","limit: 65536","entries: 72",
[...]|...]
2>  [list_to_atom("player_"++integer_to_list(Item)) || Item <- lists:seq(1,1000000) ].
[player_1,player_2,player_3,player_4,player_5,player_6,
player_7,player_8,player_9,player_10,player_11,player_12,
player_13,player_14,player_15,player_16,player_17,player_18,
player_19,player_20,player_21,player_22,player_23,player_24,
player_25,player_26,player_27,player_28,player_29|...]
3>  string:tokens( binary_to_list(erlang:system_info(info)),"\n").
["=memory","total: 93955448","processes: 37830214",
"processes_used: 37830214","system: 56125234",
"atom: 20296629","atom_used: 20279627","binary: 360656",
"code: 1965264","ets: 124260","=hash_table:atom_tab",
"size: 823117","used: 627221","objs: 1006479","depth: 7",
"=index_table:atom_tab","size: 1006592","limit: 1048576",
"entries: 1006479","=hash_table:module_code","size: 97",
"used: 54","objs: 74","depth: 4","=index_table:module_code",
"size: 1024","limit: 65536","entries: 74",
[...]|...]
4>  [list_to_atom("player_"++integer_to_list(Item)) || Item <- lists:seq(1,1000000) ].
[player_1,player_2,player_3,player_4,player_5,player_6,
player_7,player_8,player_9,player_10,player_11,player_12,
player_13,player_14,player_15,player_16,player_17,player_18,
player_19,player_20,player_21,player_22,player_23,player_24,
player_25,player_26,player_27,player_28,player_29|...]
5>  string:tokens( binary_to_list(erlang:system_info(info)),"\n").
["=memory","total: 98839096","processes: 42712630",
"processes_used: 42712630","system: 56126466",
"atom: 20296629","atom_used: 20279627","binary: 361888",
"code: 1965264","ets: 124260","=hash_table:atom_tab",
"size: 823117","used: 627221","objs: 1006479","depth: 7",
"=index_table:atom_tab","size: 1006592","limit: 1048576",    %注意再次调用的时候这里没有变化
"entries: 1006479","=hash_table:module_code","size: 97",
"used: 54","objs: 74","depth: 4","=index_table:module_code",
"size: 1024","limit: 65536","entries: 74",
[...]|...]
6>

我们挑战一下atom的数量上限, [list_to_atom("player_"++integer_to_list(Item)) || Item <- lists:seq(1,100000000) ].只要运行这个就可以了,不久我们就看到下面的提示:

所以在How to Crash Erlang 一文中,无节制使用atom名列前茅:

Run out of atoms. Atoms in Erlang are analogous to symbols in Lisp--that is, symbolic, non-string identifiers that make code more readable, like green or unknown_value--with one exception. Atoms in Erlang are not garbage collected. Once an atom has been created, it lives as long as the Erlang node is running. An easy way to crash the Erlang virtual machine is to loop from 1 to some large number, calling integer_to_list and then list_to_atom on the current loop index. The atom table will fill up with unused entries, eventually bringing the runtime system to halt.

Why is this is allowed? Because garbage collecting atoms would involve a pass over all data in all processes, something the garbage collector was specifically designed to avoid. And in practice, running out of atoms will only happen if you write code that's generating new atoms on the fly.

本文链接

[Erlang 0027] Using Record in Erlang Shell

2011-12-31T05:34:00Z

[Erlang 0006] Erlang中的record与宏中我们提到过Record是一个编译时的功能,在Erlang VM中并没有专门的数据类型.在线上解决问题有时候会遇到要在shell中使用record,那么就有两个选择:1.在shell中构造record定义,如果能构造record有了record的定义编写ets:match的匹配模式就方便多了; 2.直接使用record对应的tuple结构;

方法一使用rd命令
Eshell V5.9 (abort with ^G)
1> rd(film ,{ director, actor, type, name,imdb}).
film
2> F =#film{}.
#film{director = undefined,actor = undefined,
type = undefined,name = undefined,imdb = undefined}
3> F#film.type.
undefined
4> F#film.type=23.
* 1: illegal pattern
5> F2 =F#film{type=23}.
#film{director = undefined,actor = undefined,type = 23,
name = undefined,imdb = undefined}

方法二使用rr命令
rr命令可以加载模块中的record定义,我们把record放在一个模块中:

-module(records).
-record(book,{name, id, item01, item02, item03, item04, item05, item06, item07, item08, item09, item10 } ).
-record(film ,{ director, actor, type, name,imdb}).

我们在shell中编译并尝试加载其中的record:
Eshell V5.9 (abort with ^G)
1> c(records).
records.erl:2: Warning: record book is unused
records.erl:19: Warning: record film is unused
{ok,records}
2> rr(records).
[book,film]
3> F =#film{}.
#film{director = undefined,actor = undefined,
type = undefined,name = undefined,imdb = undefined}
4>

方法三最方便的方案 user_default

上面两种方法一种是手工定义,一种是手工加载定义,有更理想的方案么?有!
仔细阅读一下Erlang shell的文档,我们可以看到下面这段: http://www.erlang.org/documentation/doc-5.4.12/lib/stdlib-1.13.11/doc/html/shell.html

If a command (local function call) is not recognized by the shell, an attempt is first made to find the function in the module user_default, where customized local commands can be placed. If found, then the function is evaluated. Otherwise, an attempt is made to evaluate the function in the module shell_default. The module user_default must be explicitly loaded.

There is some support for reading and printing records in the shell. During compilation record expressions are translated to tuple expressions. In runtime it is not known whether a tuple actually represents a record. Nor are the record definitions used by compiler available at runtime. So in order to read the record syntax and print tuples as records when possible, record definitions have to be maintained by the shell itself. The shell commands for reading, defining, forgetting, listing, and printing records are described below. Note that each job has its own set of record definitions. To facilitate matters record definitions in the modules shell_default and user_default (if loaded) are read each time a new job is started.

我们就使用user_default来解决这个问题!!!我们准备一下测试的文件:

%% File: records.hrl
-record(book,{ name, id, item01, item02, item03, item04, item05, item06, item07, item08, item09, item10 } ).
-record(film ,{ director, actor, type, name,imdb}).
-record(foo,{ id, name,bar}).

%% File: user_default.erl
-module(user_default).
-include("records.hrl").
-compile(export_all).

get_meta() -> 
user_default:module_info().

get_timestamp() ->
{M, S, _} = erlang:now(), 
M * 1000000 + S.

编译一下user_default.erl,编译之后我们启动Erlang Shell,可以看到不仅record已经加载了,而且user_default中定义的方法也可以直接访问.

Eshell V5.9 (abort with ^G)
1> rl(). %% 在shell中可以使用rl()查看已经定义的record.
-record(book,{name, id, item01, item02, item03, item04, item05, item06, item07, item08, item09, item10 } ).
-record(film,{director,actor,type,name,imdb}).
-record(foo,{id,name,bar}).
ok
2> get_timestamp().
1325308014
3> F=foo#{}.
* 1: syntax error before: '{'
3> F=#foo{}.
#foo{id = undefined,name = undefined,bar = undefined}
4>

方法四不定义直接使用tuple
[Erlang 0006] Erlang中的record与宏我们说过record在Erlang的内部实现是还是一个tuple.在shell中我们可以使用等效的tuple.
4> F2 =#film{director=2012}.
#film{director = 2012,actor = undefined,type = undefined,
name = undefined,imdb = undefined}
5> F2 == {film , 2012, undefined, undefined, undefined, undefined} .
true

方法五 Match Specifications And Records (Dynamically!)
这是trapexit上提供的一个解决方案,稍微绕一点,主要是{ok,Tree}=epp:parse_file("myheader.hrl",["./"],[])解析头文件进行预处理,然后生成reocrd的元数据module文件,再利用工具类来动态生成match specification;

 1 %%%-------------------------------------------------------------------
 2 %%% File : record_util.erl
 3 %%% Author : Gordon Guthrie gordon@hypernumbers.com
 4 %%% Description : utilities for manipulating records
 5 %%%
 6 %%% Created : 2 Sep 2008 by Gordon Guthrie
 7 %%%-------------------------------------------------------------------
 8 -module(make_ms_util).
 9 
10 -include("myheader.hrl").
11 
12 -export([make/0]).
13 
14 -define(MODULENAME,"ms_util2").
15 
16 make() ->
17 {ok,Tree}=epp:parse_file("myheader.hrl",["./"],[]),
18 Src=make_src(Tree),
19 ok=file:write_file(?MODULENAME++".erl",list_to_binary(Src)).
20 
21 make_src(Tree) -> make_src(Tree,[]).
22 
23 make_src([],Acc) -> make_src2(Acc,[],[]);
24 make_src([{attribute,_,record,Record}|T],Acc) -> make_src(T,[Record|Acc]);
25 make_src([_H|T],Acc) -> make_src(T,Acc).
26 
27 make_src2([],Acc1,Acc2) -> top_and_tail(Acc1,Acc2);
28 make_src2([H|T],Acc1,Acc2) -> {NewAcc1,NewAcc2}=expand_rec(H),
29 make_src2(T,[NewAcc1|Acc1],[NewAcc2|Acc2]).
30 
31 expand_rec({Name,Def}) -> expand_fields(Name,Def,1,[]).
32 
33 expand_fields(Name,[],N,Acc) -> {mk2(Name,N-1),lists:reverse([mk(Name)|Acc])};
34 expand_fields(Name,[{record_field,_,{atom,_,F},_}|T],N,Acc) ->
35 expand_fields(Name,T,N+1,[mk(Name,F,N)|Acc]);
36 expand_fields(Name,[{record_field,_,{atom,_,F}}|T],N,Acc) ->
37 expand_fields(Name,T,N+1,[mk(Name,F,N)|Acc]);
38 expand_fields(Name,[H|T],N,Acc) -> expand_fields(Name,T,N+1,Acc).
39 
40 %% mk2/1 builds the no of fields fns
41 mk2(Name,N) -> "no_of_fields("++atom_to_list(Name)++") -> "++
42 integer_to_list(N)++";\n".
43 
44 %% mk/1 builds an error line
45 mk(Name) -> "get_index("++atom_to_list(Name)++",F) -> "++
46 "exit({error,\"Record: "++atom_to_list(Name)++
47 " has no field called \"++atom_to_list(F)});\n".
48 
49 mk(Name,Field,N) ->
50 "get_index("++atom_to_list(Name)++","++
51 atom_to_list(Field)++")-> "++integer_to_list(N)++";\n".
52 
53 top_and_tail(Acc1,Acc2)->
54 Top="%% This module automatically generated - do not edit\n"++
55 "\n"++
56 "%%% This module provides utilities for use in building\n"++
57 "%%% match specifications from records\n"++
58 "\n"++
59 "-module("++?MODULENAME++").\n"++
60 "\n"++
61 "-export([get_index/2,no_of_fields/1]).\n"++
62 "\n",
63 Tail1="no_of_fields(Other) -> exit({error,\"Invalid Record Name: \""++
64 "++Other}).\n\n\n",
65 Tail2="get_index(Record,_Field) -> exit({error,\""++
66 "Invalid Record Name: \"++Record}).\n",
67 Top++lists:flatten(Acc1)++Tail1++lists:flatten(Acc2)++Tail2.

详情点击这里: http://www.trapexit.org/Match_Specifications_And_Records_%28Dynamically%21%29

一个简单的问题,只要愿意去想,答案不止一个;

本文链接

[Erlang 0026] 目前最优的Erlang IDE - erlIDE

2011-12-28T09:19:00Z

[Erlang 0002] Erlang IDE从我写的一系列Erlang文章阅读人数来看,这篇文章阅读人数最多.可以窥见一斑:好多人在看是否会对一种新语言投入精力的时候,IDE是一个重要的指标,甚至可能就是因为IDE不给力而放弃一种语言,不靠谱是吧,但这是事实;写这篇文章的时候erlide插件还比较差,我提到它的几个问题:无代码自动完成,偶尔提示一下也特别慢,最关键的一点:不支持中文,就因为这个无论它多么好直接被否定掉了; 但是作为少数几个还在持续改进的Erlang IDE插件,erlide现在的版本已经堪称是目前最好用的Erlang IDE.

体验了一下,目前的版本:

完美支持中文,在代码中输入中文会提示你把文件编码更改到UTF-8
已经有了代码自动完成,包括自定义模块的自动完成
方法转到定义功能不错
打开大文件(约1MB)没有压力
保存即编译

几点注意事项:

安装

作为Eclipse插件首先下载一个Eclipse Classic最新版本,我下载的是Eclipse Classic 3.7.1,安装完成之后按照,插件主页:http://erlide.org/上的说明即可:

Install Erlide by going to Help → Install new software... . In the dialog, enter http://erlide.org/update as URL. Follow the dialogs with the obvious choices and in the end you will be asked to restart.

An alternative way to install is from the Eclipse marketplace Help → Eclipse Marketplace... or if you are using Eclipse 3.7 simply by dragging and dropping the button below into a running Eclipse workspace

Restart. Go to Window → Preferences → Erlang → Installed runtimes and add an entry (or several) for your Erlang installation(s) of choice. The required parameters are the name and the path to the top level directory (i.e. $ERL_TOP). Now restart again.

版本

你可能担心支持最15B版本的Erlang么? 答案:支持,在Eclipse中打开Window → Preferences → Erlang,可以选择当前IDE使用哪个版本的Erlang .见下图

写的技术文章过时了不郁闷,郁闷的是可以更新自己的博客却更新不了被抓站程序抓走的

本文链接

博客园_回头再说 坚强2002的博客

[Erlang 0035] Erlang SMP

[Erlang 0034] Erlang iolist

[Erlang 0033] 接入Erlang控制台的几种方法

[Erlang 0032] Erlang Binary的内部实现

[Erlang 0031] Erlang Shell中的输出完整数据

[Erlang 0030] 理解Erlang/OTP Supervisor

[Erlang 0029] Erlang Inline编译

[Erlang 0028] Erlang atom

[Erlang 0027] Using Record in Erlang Shell

[Erlang 0026] 目前最优的Erlang IDE - erlIDE

博客园_回头再说坚强2002的博客