CloudCrowd ― Parallel Processing for the Rest of Us
Rubyによる並列処理システム「CloudCrowd 0.1.0」リリース
ファミコン的なロゴとかアーキテクチャの図とかゆるい感じです。
インストール
# sudo gem install cloud-crowd Password: Building native extensions. This could take a while... Successfully installed rack-1.0.0 Successfully installed sinatra-0.9.4 Successfully installed activesupport-2.3.4 Successfully installed activerecord-2.3.4 Successfully installed json-1.1.9 Successfully installed rest-client-1.0.3 Successfully installed right_http_connection-1.2.4 Successfully installed right_aws-1.10.0 Successfully installed cloud-crowd-0.1.0 9 gems installed Installing ri documentation for rack-1.0.0... Installing ri documentation for sinatra-0.9.4... Installing ri documentation for activesupport-2.3.4... Installing ri documentation for activerecord-2.3.4... Installing ri documentation for json-1.1.9... Installing ri documentation for rest-client-1.0.3... Installing ri documentation for right_http_connection-1.2.4... Installing ri documentation for right_aws-1.10.0... Installing ri documentation for cloud-crowd-0.1.0... Installing RDoc documentation for rack-1.0.0... Installing RDoc documentation for sinatra-0.9.4... Installing RDoc documentation for activesupport-2.3.4... Installing RDoc documentation for activerecord-2.3.4... Installing RDoc documentation for json-1.1.9... Installing RDoc documentation for rest-client-1.0.3... Installing RDoc documentation for right_http_connection-1.2.4... Installing RDoc documentation for right_aws-1.10.0... Installing RDoc documentation for cloud-crowd-0.1.0...
help
# crowd --help
CloudCrowd is a MapReduce-inspired Parallel Processing System for Ruby.
Wiki: http://wiki.github.com/documentcloud/cloud-crowd
Rdoc: http://rdoc.info/projects/documentcloud/cloud-crowd
Usage: crowd COMMAND OPTIONS
Commands:
install Install the CloudCrowd configuration files to the specified directory
server Start up the central server (requires a database)
workers Control worker daemons, use: (start | stop | restart | status | run)
console Launch a CloudCrowd console, connected to the central database
load_schema Load the schema into the database specified by database.yml
Options:
-c, --config PATH path to configuration directory
-n, --num-workers NUM number of worker processes
-p, --port PORT central server port number
-e, --environment ENV server environment (sinatra)
-v, --version show version
configurationファイルをつくる
# crowd install firstcrowd/config installed firstcrowd/config/config.yml installed firstcrowd/config/config.ru installed firstcrowd/config/database.yml installed firstcrowd/config/actions
database.ymlで定義するDBを作成する
スキーマを作成
# crowd load_schema
-- create_table("jobs", {:force=>true})
-> 0.8320s
-- create_table("work_units", {:force=>true})
-> 0.0364s
-- create_table("worker_records", {:force=>true})
-> 0.0073s
-- add_index("jobs", ["status"], {:name=>"index_jobs_on_status"})
-> 0.2406s
-- add_index("work_units", ["job_id"], {:name=>"index_work_units_on_job_id"})
-> 0.0394s
-- add_index("work_units", ["status", "worker_record_id", "action"], {:name=>"index_work_units_on_status_and_worker_record_id_and_action"})
-> 0.0474s
-- add_index("worker_records", ["name"], {:name=>"index_worker_records_on_name"})
-> 0.1785s
-- add_index("worker_records", ["updated_at"], {:name=>"index_worker_records_on_updated_at"})
-> 0.0850s
-- initialize_schema_migrations_table()
-> 0.0128s
-- assume_migrated_upto_version(1)
-> 0.0146s
CentralServer用のthinをインストール
# sudo gem install thin Password: Building native extensions. This could take a while... Building native extensions. This could take a while... Successfully installed eventmachine-0.12.8 Successfully installed thin-1.2.4 2 gems installed Installing ri documentation for eventmachine-0.12.8... Installing ri documentation for thin-1.2.4... Installing RDoc documentation for eventmachine-0.12.8... Could not find main page README Could not find main page README Could not find main page README Could not find main page README Installing RDoc documentation for thin-1.2.4...
CentralServerを起動
# crowd server >> Thin web server (v1.2.4 codename Flaming Astroboy) >> Maximum connections set to 1024 >> Listening on 0.0.0.0:9173, CTRL+C to stop
CentralServerが起動した
workersを起動
# crowd workers start -n 5 #
workersが起動している
なんか処理させよう・・
