kk Blog —— 通用基础


date [-d @int|str] [+%s|"+%F %T"]
netstat -ltunp
sar -n DEV 1

octopress优化

octopress优化

能够让octopress在50篇文章下跑进5s，不优化要跑60s左右
300篇15s左右

运行过程

看octopress目录下的Rakefile，里面有generate，preview，watch等。

输入rake generate是就是按照Rakefile中task :generate do执行的。

最主要的两条：

1
2

system "compass compile --css-dir #{source_dir}/stylesheets"
system "jekyll build"

第一条是编译css，第二条是生成文章。

第一条不知道如何优化，略过。

第二条接着执行到

/usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/commands/build.rb

文件的 init_with_program -> process -> build 然后到

/usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/command.rb 文件的 process_site 然后到

/usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb 文件的 process，

process代码如下：

1
2
3
4
5
6
7
8

47     def process
48       reset
49       read
50       generate
51       render
52       cleanup
53       write
54     end

这里的函数都在这个文件里，主要费时在 generate 和 render。

1. generate

generate 会执行 octopress/plugins 目录下的 tag_generator.rb 和 category_generator.rb，

这两个文件的 write_tag_indexes 和 write_category_indexes 分别回构建 tag 和 category 的分类首页。

所以 tag 和 category 越多构建越慢。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

from /home/kk/kk/github/octopress/plugins/rubypants.rb:261:in `to_html'
from /home/kk/kk/github/octopress/plugins/octopress_filters.rb:31:in `post_filter'
from /home/kk/kk/github/octopress/plugins/octopress_filters.rb:41:in `post_render'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:255:in `block in post_render'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:254:in `each'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:254:in `post_render'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:224:in `do_layout'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/page.rb:122:in `render'
from /home/kk/kk/github/octopress/plugins/category_generator.rb:100:in `write_category_index'
from /home/kk/kk/github/octopress/plugins/category_generator.rb:112:in `block in write_category_indexes'
from /home/kk/kk/github/octopress/plugins/category_generator.rb:111:in `each'
from /home/kk/kk/github/octopress/plugins/category_generator.rb:111:in `write_category_indexes'
from /home/kk/kk/github/octopress/plugins/category_generator.rb:141:in `generate'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb:280:in `block in generate'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb:279:in `each'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb:279:in `generate'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb:50:in `process'

2. render

1
2
3
4
5
6
7
8
9
10
11

from /home/kk/kk/github/octopress/plugins/rubypants.rb:261:in `to_html'
from /home/kk/kk/github/octopress/plugins/octopress_filters.rb:31:in `post_filter'
from /home/kk/kk/github/octopress/plugins/octopress_filters.rb:41:in `post_render'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:249:in `block in pre_render'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:248:in `each'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:248:in `pre_render'
from /usr/local/lib/ruby/gems/2.1.0/gems/octopress-hooks-2.2.1/lib/octopress-hooks.rb:222:in `do_layout'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/page.rb:122:in `render'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb:299:in `block in render'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb:298:in `each'
from /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb:298:in `render'

都是慢在 octopress/plugins/rubypants.rb 的 to_html 函数。

优化

1、to_html 函数的 tokenize 和 gsub 很慢。

to_html 函数只是把一些' “之类的转成html，试了一下不执行to_html，diff出来差别不大，页面上显示也都还OK。单纯的 < 在其他地方已经转好了。

所以 octopress/plugins/octopress_filters.rb 中不执行 to_html

2、每次计算侧边栏太慢

所有页面都是按照其 layout 的格式找对应 source/_layout/ 下的模板来生成的。

注意 post.html 和 page.html 的最后都有 { { include_array XXX } }, 这就是说每个页面都要运行 plugins/include_array.rb 中的 render(context) 来生成侧边栏。

但是侧边栏应该是(?)都一样的。所以改成隔一定时间计算一次

1
2
3
4
5
6
7
8
9

 26     @@caltime = 0
 27     @@retstore = ""
 28     def render(context)
 29         if Time.now.to_f - @@caltime < context.registers[:site].config['recaltime']
 30                 return @@retstore;
 31         end
 32         @@caltime = Time.now.to_f
...
 58       @@retstore = rtn

为什么要隔一段时间？因为在 preview 中有改动任何文件就会重新生成一次，这时侧边栏也要重新计算

3、减小文件大小

我的侧边栏有二级目录，整个侧边栏比较大，50篇时已经有20k。所以将侧边栏独立成一个文件，用js来load。

plugins/include_array.rb 中加：

1
2
3
4
5

59       fp = File.new('sidebar.html', 'w');
60       fp.puts(rtn);
61       fp.puts('<script src="/javascripts/category.js" type="text/javascript"></script>');
62       fp.puts('<script type="text/javascript"> hadOpenDiv();</script>');
63       fp.close();

source/_layout/post.html和source/_layout/page.html 的最后部分改成：

1
2
3
4
5

35 <aside class="sidebar" id='load_sidebar'>
36 </aside>
37 <script type="text/javascript">
38   $('#load_sidebar').load('/sidebar.html');
39 </script>

因为 source/index.html 里也有 { { include_array XXX } }，所以不用担心不执行 plugins/include_array.rb。

问题1：就是"最近评论" 要从 _config.yml 中的 default_asides: 中移到 source/index.html 中。因为js load进的文件中的 document.write 不执行了。这也就是为什么 source/index.html 不采用js load。如果是 <aside> 中增加 div 用来 load 的话，侧边栏缩到底部就没有三列的效果。

问题2：sidebar.html 要写到主目录，source/sidebar.html 中用 ln 软链接到 sidebar.html。因为如果在watch状态直接写到 source/ 目录的话，他就会再次认为文件有改变，又重新生成。造成在不断生成的死循环。细节是在Rakefile文件的 generate 和 preview 中先加软链接再运行其他

1
2
3

56 task :generate do
57   system('>`pwd`/sidebar.html')
58   system('ln -f -s `pwd`/sidebar.html `pwd`/source/sidebar.html')

优化效果

在文件中加入时间输出代码

1

p "    #{ __FILE__} line:#{__LINE__} time:#{Time.now.to_f}"

50篇文章，40个tag，61个category，运行如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

kk@kk-laptop:~/kk/github/octopress(source)$ rake generate
## Generating Site with Jekyll
"/home/kk/kk/github/octopress/Rakefile line:62 time:1416933869.1204205"
identical source/stylesheets/screen.css 
"/home/kk/kk/github/octopress/Rakefile line:64 time:1416933870.019113"
Configuration file: /home/kk/kk/github/octopress/_config.yml
	        Source: source
	   Destination: public
	  Generating... 
"    /home/kk/kk/github/octopress/plugins/category_generator.rb line:110 time:1416933870.7509217"
"    /home/kk/kk/github/octopress/plugins/category_generator.rb line:114 time:1416933871.8838022"
"    /home/kk/kk/github/octopress/plugins/tag_generator.rb line:93 time:1416933871.8838577"
"    /home/kk/kk/github/octopress/plugins/tag_generator.rb line:97 time:1416933872.389736"
"    /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb line:297 time:1416933872.4271524"
"    /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb line:301 time:1416933873.8506286"
	                done.
 Auto-regeneration: disabled. Use --watch to enable.
"/home/kk/kk/github/octopress/Rakefile line:66 time:1416933873.9671116"
"total = 4.846851825714111"

total = 4.846851825714111，不优化要60s左右。

300篇

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

kk@kk-laptop:~/kk/github/octopress(source)$ rake generate
## Generating Site with Jekyll
"/home/kk/kk/github/octopress/Rakefile line:62 time:1426179501.0918927"
identical source/stylesheets/screen.css 
"/home/kk/kk/github/octopress/Rakefile line:64 time:1426179502.0703895"
Configuration file: /home/kk/kk/github/octopress/_config.yml
	        Source: source
	   Destination: public
	  Generating... 
"    /home/kk/kk/github/octopress/plugins/category_generator.rb line:110 time:1426179503.376222"
"    /home/kk/kk/github/octopress/plugins/category_generator.rb line:114 time:1426179508.2033086"
"    /home/kk/kk/github/octopress/plugins/tag_generator.rb line:93 time:1426179508.2033658"
"    /home/kk/kk/github/octopress/plugins/tag_generator.rb line:97 time:1426179508.3304708"
"    /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb line:297 time:1426179508.3851612"
"    /usr/local/lib/ruby/gems/2.1.0/gems/jekyll-2.5.1/lib/jekyll/site.rb line:301 time:1426179515.877446"
	                done.
 Auto-regeneration: disabled. Use --watch to enable.
"/home/kk/kk/github/octopress/Rakefile line:66 time:1426179516.2085445"
"total = 15.116710424423218"

Posted by kk blog, octopress

« octopress 加站长统计和站内搜索 jekyll的一些全局变量 »