_.iteratee = function(value, context, argCount) {}

1
2
3
4
5
6
7
8
9
// A mostly-internal function to generate callbacks that can be applied
// to each element in a collection, returning the desired result — either
// identity, an arbitrary callback, a property matcher, or a property accessor.
_.iteratee = function(value, context, argCount) {
  if (value == null) return _.identity;
  if (_.isFunction(value)) return createCallback(value, context, argCount);
  if (_.isObject(value)) return _.matches(value);
  return _.property(value);
};

_.iteratee的作用是生成callbacks,应用于collection的每个元素,返回分4种情况。

1) value == null

return _.identity

1
2
3
_.identity = function(value) {
  return value;
};

Example

1
2
3
4
> .identity('input value')
<- "input value"
> .iteratee(null)('input value')
<- "input value"

2) _.isFunction(value)

如果value为function, bind value(func) with context。 调用createCallback(value, context, argCount),return function(value, index, collection) {...}

3) _.isObject(value)

return _.matches(value) returns a predicate function.

4) else

return _.property(value)

Read on →

Yum Install PG9.4

1
2
3
4
5
wget http://yum.postgresql.org/9.4/redhat/rhel-6-x86_64/pgdg-redhat94-9.4-1.noarch.rpm
yum instlal pgdg-redhat94-9.4-1.noarch.rpm
yum install postgresql94-server
service postgresql-9.4 initdb
chkconfig postgresql-9.4 on

Backup Data

1
2
su - postgres
pg_dumpall > dump.sql

Restore Data

1
2
3
4
service postgresql stop
service postgresql-9.4 start
su - postgres
psql < dump.sql

Config Network Access

vi /var/lib/pgsql/9.4/data/postgresql.conf

1
2
listen_addresses = '*'
port = 5432

/var/lib/pgsql/9.4/data/pg_hba.conf

1
2
3
4
5
6
7
8
# "local" is for Unix domain socket connections only
local   all         all                               ident
# IPv4 local connections:
host    all         all         127.0.0.1/32          ident
host    all         all         130.51.79.0/24        md5
host    all         all         10.210.29.0/24        md5
# IPv6 local connections:
host    all         all         ::1/128               ident

Remove PG8.4

1
2
yum remove postgresql
ln -s /usr/pgsql-9.4/bin/psql /usr/local/bin/psql

Refrences

  1. http://yum.postgresql.org/repopackages.php#pg94
  2. https://wiki.postgresql.org/wiki/YUM_Installation
  3. http://www.postgresql.org/docs/9.4/static/upgrading.html

1. Bundle a new gem

1
bundle gem jruby_gem

edit jrubygem.gemspec

1
2
3
4
Gem::Specification.new do |spec|
  ...
  spec.adddependency "jbundler"
end

Add TestUnit

1) edit jruby_gem.gemspec

1
2
3
4
Gem::Specification.new do |spec|
  ...
  spec.add_development_dependency "minitest"
end

2) edit RakeFile

Add test task code as below.

1
2
3
4
5
6
7
8
9
10
...
require 'rake/testtask'

Rake::TestTask.new do |t|
  t.libs << 'test'
  t.pattern = "test/*_test.rb"
end

desc "Run tests"
task :default => :test

3) create test/test_helper.rb

1
2
3
4
5
require 'jruby_gem'

require 'minitest/autorun'
require 'minitest/unit'
require 'minitest/pride'

Add Jar Dependency

edit jruby_gem.gemspec

1
2
3
4
Gem::Specification.new do |spec|
  ...
  spec.requirements << "jar 'jar-name', 'jar-version'"
end

Add Initializer

edit jrubygem/jrubygem.rb

2. In Rails App

edit Gemfile

1
gem 'jruby_gem', path: 'xxx'

1
2
3
touch Jarfile

jbunle

jbundle will read the requirements of jruby_gem.gemspec and then configure the classpath

Reference

  1. http://blakewilliams.me/blog/developing-gems-with-tdd-and-minitest-pt-1
  2. https://github.com/mkristian/jbundler/wiki/Gem-with-declared-jars
  3. https://github.com/richfisher/jruby_activiti

1. About Monit

Monit is a small Open Source utility for managing and monitoring Unix systems. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.

2. Install Monit

1
2
3
yum install monit
chkconfig monit on
service monit start

3. Configure Monit Web Server

/etc/monit.d/monitrc

1
2
set httpd port 2812
  allow localhost

4. Configurations

1
2
3
4
5
6
7
8
check process httpd
  with pidfile /etc/httpd/run/httpd.pid
  start program = "/etc/init.d/httpd start"
  stop program = "/etc/init.d/httpd stop"
  if failed host localhost port 8081
  protocol http then restart
  if totalmem > 1024 MB for 2 cycles then restart
  if 5 restarts within 5 cycles then timeout
1
2
3
4
5
6
check process sidekiq
  with pidfile /var/your_app/shared/tmp/pids/sidekiq.pid
  start program = "/bin/bash -l -c 'cd /var/your_app/current && bundle exec sidekiq --index 0 --pidfile /var/your_app/shared/tmp/pids/sidekiq.pid --environment production --logfile /var/your_app/shared/log/sidekiq.log --daemon'"
  stop program = "/bin/bash -l -c 'cd /var/your_app/current && bundle exec sidekiqctl stop /var/your_app/shared/tmp/pids/sidekiq.pid 10'"
  if totalmem is greater than 512 MB for 3 cycles then restart
  if 3 restarts within 5 cycles then timeout
1
2
3
4
5
6
check process mysqld
  with pidfile /var/run/mysqld/mysqld.pid
  start program = "/etc/init.d/mysqld start"
  stop program = "/etc/init.d/mysqld stop"
  if failed host 127.0.0.1 port 3306 then restart
  if 5 restarts within 5 cycles then timeout
1
2
3
4
5
6
check process redis
  with pidfile /var/run/redis/redis.pid
  start program = "/etc/init.d/redis start"
  stop program = "/etc/init.d/redis stop"
  if failed host 127.0.0.1 port 6379 then restart
  if 5 restarts within 5 cycles then timeout

5. Reference

http://mmonit.com/monit/

http://railscasts.com/episodes/375-monit

1. Get Start

1
gem install selenium-webdriver

2. Basics Usage

1
2
@driver=Selenium::WebDriver.for :firefox
@wait = Selenium::WebDriver::Wait.new(:timeout => 10)
Navigate
1
@driver.navigate.to 'http://xx.xx.xx.xx'
Switch
1
2
3
4
5
@driver.switch_to.default_content

@driver.switch_to.frame('frame_id')

@driver.switch_to.window('window_id')
Find element
1
2
3
4
5
@driver.find_element(id: 'xxx')
@driver.find_element(name: 'xxx')
@driver.find_element(tag_name: 'xxx')
@driver.find_element(xpath: 'xxx')
@driver.find_element(css: 'xxx')
Click
1
2
el = @driver.find_element()
el.click
Double Click
1
2
el = @driver.find_element()
@driver.action.double_click(el).perform

3. Input

Text
1
2
find_displayed_element(name: 'element_name').clear
find_displayed_element(name: 'element_name').send_keys('123')
Select
1
2
3
4
el = find_displayed_element(name: field_name)
el.find_elements(tag_name: 'option').each_with_index do |option, index|
  option.click if option.text == 'target_text'
end

4. Common functions

1
2
3
4
5
def find_displayed_element(option)
  @wait.until { @driver.find_element(option) }
  @wait.until { @driver.find_element(option).displayed? }
  @driver.find_element(option)
end
1
2
3
4
5
6
def switch_to_displayed_frame(frame_id)
  find_displayed_element(id: frame_id)
  @driver.switch_to.frame(frame_id)

  @driver.execute_script("window.showModalDialog = window.open;");
end
1
2
3
4
5
6
7
8
9
def get_window_id_by_title(window_title)
  @wait.until { @driver.window_handles.size > 1 }

  wnd_titl = @driver.window_handles.map do |w|
    @driver.switch_to.window(w)
    [w, @driver.title]
  end
  win_id = wnd_titl.find{|e1,e2| e2 == window_title }.first
end

5. References

  1. http://docs.seleniumhq.org/
  2. https://code.google.com/p/selenium/wiki/RubyBindings

1. 下载

instant client 32bit www.oracle.com/technetwork/cn/topics/winsoft-085727.html‎

instant client 64bit www.oracle.com/technetwork/topics/winx64soft-089540.html

2. 解压

D:/instantclient121_32bit

D:/instantclient121_64bit

3. 添加环境变量

TNSADMIN D:/instantclient12164bit

4. 配置PL/SQL

D:/instantclient121_32bit/TNSNAMES.ORA

1
2
3
4
5
6
7
8
9
10
your_config_name =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = your_server_ip)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SID = your_sid)
      (SERVER = DEDICATED)
    )
  )

PL/SQL->Tools->Perferences->Oracle->Connection

Oracle Home=D:/instantclient12132bit OCILibrary=D:/instantclient121_32bit/oci.dll

5. 配置Navicat(64bit)

Navicat自带的oracle客户端有问题,需要手工指定路径。

Tools->Options->OCI

D:\instantclient121_64bit\oci.dll

WARNING: The following is the study notes of "MetaProgramming Ruby". As the only academic research, if you like the content, please support genuine.

以下内容为《Ruby元编程》的读书笔记,仅作为学术研究,如果您喜欢以下内容,请支持正版.

I. The object model

Open Class

1
2
3
4
5
class String
  def to_alphanumeric
    gsub /[^\w\s]/,''
  end
end

Inside Class Definitions

1
2
3
4
5
3.times do
  class C
    puts 'hello'
  end
end

class关键字把你带到类的上下文,让你可以在其中定义方法。

What's in an Object

1
2
3
4
5
6
7
8
class MyClass
  def my_method
    @v = 1
  end
end

obj = MyClass.new
obj.class # => MyClass

实例变量

1
2
obj.my_method
obj.instance_variabes  #=> [:@v]

方法

1
obj.methods.grep(/my/) # => [:my_method]

一个对象仅包含它的实例变量和一个对自身类的引用。

方法存放在类中,而非对象中。

1
2
String.instance_methods == "abc".methods  # => true
String.methods == "abc".methods  # => false

Classes Revisited

类自身也是对象。类也有自己的类,名为Class。类的方法就是Class的实例方法。

1
2
"abc".class # => String
String.class # => Class
1
2
3
String.superclass # => Object
Object.superclass # => BasicObject
BasicObject.superclass # => nil
1
2
Class.superclass # => Module
Module.superclass # => Object
1
2
class MyClass; end
obj1 = MyClass.new

obj1 和 MyClass 都是引用,唯一区别在于,obj1是一个变量,而MyClass是一个常量。

Alt text

Read on →

Mechanize, 一个ruby类库,可以对网站进行交互,如抓取、登录等; OCR, 验证码识别神器.

1. 简单的用户名和密码登陆,无验证码

1
2
3
4
5
@agent = Mechanize.new
form = @agent.get("login_url")
form.username = 'username'
form.password = 'password'
form.submit

2. 带验证码的登陆

首先安装神器 tesseract

1
2
brew install tesseract
gem install tesseract-ocr

不一定能一次识别到,所以通过一个loop直到找到4位数字验证码。

1
2
images = page.search('.checkcode img')
vali_code = get_vali_code(images.first.attributes["src"])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def get_vali_code(src)
  code = ''
  loop do
    image_string = @agent.get(src).body_io.string

    e = Tesseract::Engine.new {|e|
      e.language  = :eng
      e.blacklist = '|'
    }

    code = e.text_for(image_string).strip.match(/\d+/).to_s

    break if code.size == 4
  end
  code
end

3. 一些用到的

Get and post

1
2
page1 = @agent.get(url)
page2 = @agent.post(url, params)

find element

1
page.search('.checkcode img')

current url

1
page.uri.to_s

操作链接

遍历

1
page.links.each {|link| }

获取地址

1
link.href

点击

1
link.click

4. Reference

http://asciicasts.com/episodes/191-mechanize https://github.com/meh/ruby-tesseract-ocr

1. Background

通过Newrelic监控到某个子应用SQL平均执行时间达到了300ms,痛点如下:

1
SELECT COUNT(*) FROM `table_rows` WHERE `table_rows`.`table_id` = 1 AND `table_rows`.`user_id` = 98

已经分别为tableid和userid建立索引。

EXPLAIN EXTRA结果:

1
Using intersect(index_table_rows_on_user_id,index_table_rows_on_table_id); Using where

2. Solutions

建立联合索引

1
rails g migration add table_id_and_user_id_index_to_table_rows
1
2
3
4
5
class AddTableIdAndUserIdIndexToTableRows < ActiveRecord::Migration
  def change
    add_index :table_rows, [:table_id, :user_id]
  end
end
1
rake db:migrate

执行后,开发环境查询时间由40ms提高到8ms。

EXPLAIN EXTRA

1
Using index

will_paginate

对于总量特别大的,可以为willpaginate设置totalentries,以避免count操作。

1
scope.paginate(:page=>params[:page], :total_entries=>1000)

3. Troubleshooting

deploy至production. 测试了一下,查询时间依然没有优化。

EXPLAIN一下

本地EXTRA: Using index

production EXTRA: Using where; Using index

本地mysql版本为5.6.13, production的版本为5.5.34,于是将production的mysql-server升级至5.6。问题解决!

4. Summary

  1. 建立合适的index。
  2. 将mysql-server升级至5.6
  3. 通过where减少scan的范围。

if you have query like SELECT COUNT(*) FROM USER It will be much faster for MyISAM (MEMORY and some others) tables because they would simply read number of rows in the table from stored value. Innodb will however need to perform full table scan or full index scan because it does not have such counter, it also can’t be solved by simple singe counter for Innodb tables as different transactions may see different number of rows in the table.

If you have query like SELECT COUNT(*) FROM IMAGE WHERE USER_ID=5 this query will be executed same way both for MyISAM and Innodb tables by performing index rage scan. This can be faster or slower both for MyISAM and Innodb depending on various conditions.

So remember Innodb is not slow for ALL COUNT() queries but only for very specific case of COUNT() query without WHERE clause.

References

http://www.mysqlperformanceblog.com/2006/12/01/count-for-innodb-tables/

DB

Backup all database.

mysqldump -uroot -p –all-database > 2014.sql mv /var/lib/mysql /var/lib/mysql.bak

Remove old mysql, mysql-server

yum list installed | grep -i mysql

yum remove mysql mysql-*

Download Yum Repository RPM Package

http://dev.mysql.com/downloads/repo/

Install RPM Package

rpm -ivh mysql-community-release-el5-5.noarch.rpm

Install mysql, mysql-server

yum install mysql mysql-server mysql-devel service mysqld start chkconfig mysqld on

Setting Password

mysqladmin -u root password

Import backup data

mysql -u root -p < ./2014.sql