# es2unix **Repository Path**: mrytsr/es2unix ## Basic Information - **Project Name**: es2unix - **Description**: Command-line ES - **Primary Language**: Unknown - **License**: Apache-2.0 - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2015-07-10 - **Last Updated**: 2024-06-14 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # es2unix **Deprecated since elasticsearch 1.0. Use the [cat API instead](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cat.html). Also check out https://github.com/drewr/copycat.** Elasticsearch API consumable by the command line. JSON isn't always the most convenient output, particularly on a terminal. The tabular format has stuck around for good reason. It's compact. It's line-oriented. es2unix strives to keep spaces significant so all output works with existing *NIX tools. `grep`, `sort`, & `awk` are first-class citizens here. # Install es2unix's only dependency is Java (Oracle or OpenJDK). Version 7 should be preferred, but there is no functional difference with 6. Earlier versions aren't supported. curl -s download.elasticsearch.org/es2unix/es >~/bin/es chmod +x ~/bin/es You can also pin to a particular version from your provisioning tools. curl -s download.elasticsearch.org/es2unix/es-20130503595fce2 >~/bin/es # Usage The `es` command takes subcommands and a few options. It assumes it's talking to ES at its default HTTP port using `http://localhost:9200` but accepts `-u` to change that. It must be a fully qualifed URL with scheme, host, & port. You can also supply `-v`, for most commands, to print a column header. ## Version % es version es 1.0.0 elasticsearch 0.21.0.Beta1 ## Health % es health -v cluster status nodes data pri shards relo init unassign kluster green 2 2 3 6 0 0 0 ## Count Sometimes you need a quick count to tell whether a cluster has any data and whether it's indexing. You can also supply a query. % es count 2,319,799 % es count elasticsearch 3 "q=elasticsearch" % es count "john deacon" 225,839 "q=john deacon" % es count "\"saved by the bell\"" 220 "q="saved by the bell"" ## Search Not exhaustive access to the query API by any stretch, but it suffices when you need to get a glance of the data in your cluster. Searches across indices with a default query of `*:*`. % es search 1.0 wiki page 1228929 1.0 wiki page 1229142 1.0 wiki page 1229146 1.0 wiki page 1229153 1.0 wiki page 1228943 1.0 wiki page 1229155 1.0 wiki page 1228950 1.0 wiki page 1229159 1.0 wiki page 1228956 1.0 wiki page 1229160 Total: 2319799 Can also specify a query, like `es search \"george costanza\"`, and, possibly more interestingly, a list of fields to return. % es search -v "george costanza" title score index type id title 5.78647 wiki page 660183 George Costansa 5.78647 wiki page 273868 George Constanza 5.63803 wiki page 865781 Vandelay Industries 4.69835 wiki page 932333 Art Vandelay 4.69835 wiki page 2147975 Can't Stand Ya 4.67351 wiki page 2486208 Art vandelay 4.07630 wiki page 2147959 Costanza 3.23200 wiki page 2147971 The Costanza family 3.21007 wiki page 2147972 Costanza family 2.94863 wiki page 4946953 Santa costanza Total: 118186 ## Master % es master J-erllamTOiW5WoGVUd04A 127.0.0.1 Slade, Frederick ## Indices % es indices -v status name pri rep docs size green _river 0 1 4 8068 green wiki 1 1 1104894 13805525784 Maybe your cluster is red and you need to know which indices are affected: % es indices | grep \^red red bb 5 0 red test 4 1 218b 218 0 red enron 5 0 red uno 1 0 ## Allocation Displays shard allocation counts across nodes % es allocation -v count ip name 12 192.168.0.24 Sage 30 x.x.x.x UNASSIGNED ## Nodes What HTTP port is `Cannonball I` listening on? Who's the master? Who's master-eligible? Who's got `data=true`? % es nodes Uv1Iy8FvR0y6_RzPXKBolg 127.0.0.1 9201 127.0.0.1 9300 d Cannonball I J-erllamTOiW5WoGVUd04A 127.0.0.1 9200 127.0.0.1 9301 * d Slade, Frederick j27iagsmQQaeIpl6yU6mCg 127.0.0.1 9203 127.0.0.1 9303 - c Georgianna Castleberry T1aFDU2BSUm748gYxjEN9w 127.0.0.1 9202 127.0.0.1 9302 d Living Tribunal If you have access to logs from all the nodes, you can run `lifecycle` to get a playback of all the node joinings and leavings with their timestamps ordered sequentially. This is much faster than combing the logs and piecing together the sequence manually. % es lifecycle /tmp/es-*/logs/elasticsearch.log 2013-02-08 13:47:15,516 Lurking Unknown INIT 0.21.0.Beta1-SNAPSHOT 2013-02-08 13:47:20,413 Lurking Unknown MASTER Lurking Unknown 2013-02-08 13:47:20,467 Lurking Unknown START 2013-02-08 13:47:36,319 Cameron Hodge INIT 0.21.0.Beta1-SNAPSHOT 2013-02-08 13:47:41,211 Lurking Unknown ADD Cameron Hodge 2013-02-08 13:47:41,223 Cameron Hodge MASTER Lurking Unknown 2013-02-08 13:47:41,278 Cameron Hodge START 2013-02-08 13:47:59,426 Armageddon INIT 0.21.0.Beta1-SNAPSHOT 2013-02-08 13:48:04,279 Lurking Unknown ADD Armageddon 2013-02-08 13:48:04,280 Cameron Hodge ADD Armageddon 2013-02-08 13:48:04,287 Armageddon MASTER Lurking Unknown 2013-02-08 13:48:04,340 Armageddon START 2013-02-08 13:48:30,333 Lurking Unknown REMOVE Armageddon 2013-02-08 13:48:30,339 Cameron Hodge REMOVE Armageddon 2013-02-08 13:48:30,362 Armageddon STOP ## Heap Heap across the cluster. % es heap | sort -rnk6 XO6c2A1D 23.9mb 25138608 123.7mb 129761280 19.4% 127.0.0.1 Junkpile uVP8g9_l 94.6mb 99257976 990.7mb 1038876672 9.6% 127.0.0.1 Hammond, Jim pjbeg_k8 76.9mb 80730208 990.7mb 1038876672 7.8% 127.0.0.1 Scarlet Centurion For some quick and dirty monitoring, I like to put this in a loop. % while true; do es heap | sort -rnk6 | head -1; sleep 60; done XO6c2A1D 57.3mb 60157200 123.7mb 129761280 46.4% 127.0.0.1 Junkpile XO6c2A1D 54.7mb 57405904 123.7mb 129761280 44.2% 127.0.0.1 Junkpile XO6c2A1D 62.7mb 65834752 123.7mb 129761280 50.7% 127.0.0.1 Junkpile XO6c2A1D 56.9mb 59743504 123.7mb 129761280 46.0% 127.0.0.1 Junkpile XO6c2A1D 52.1mb 54676216 123.7mb 129761280 42.1% 127.0.0.1 Junkpile XO6c2A1D 37.1mb 38971744 123.7mb 129761280 30.0% 127.0.0.1 Junkpile XO6c2A1D 52mb 54528424 123.7mb 129761280 42.0% 127.0.0.1 Junkpile XO6c2A1D 46.5mb 48787064 123.7mb 129761280 37.6% 127.0.0.1 Junkpile This can be extremely helpful during indexing, for example. If you see a single node showing up a lot, you might have hot shard(s) there. If you see all the nodes regularly showing up with varying heap usage percentage, it's likely a healthy cluster with good shard distribution. Searching has slightly different characteristics, but you can make similarly helpful inferences. ## Ids Sometimes it's helpful to retrieve the ids of all documents in an ES index. % es ids test -v index type id test doc 1 test doc 2 test doc 3 test doc 4 ## Shards ### Node startup We've started up three nodes where we had two before. ES decided to move one shard to the third node. % es shards wiki 0 p STARTED 1160290 7.2gb 7776371641 127.0.0.1 Feline wiki 0 r STARTED 1160290 7.2gb 7776371602 127.0.0.1 Jenkins, Abner wiki 1 p RELOCATING 1159509 7.5gb 8116295811 127.0.0.1 Feline -> 127.0.0.1 Amphibius wiki 1 r STARTED 1159509 7.5gb 8116295811 127.0.0.1 Jenkins, Abner ### After turning on more replicas We set `index.number_of_replicas` to `2`, so ES is creating another copy of each primary shard. % es shards wiki 0 p STARTED 1160290 7.2gb 7776371641 127.0.0.1 Feline wiki 0 r INITIALIZING 0 100.2mb 105077522 127.0.0.1 Amphibius wiki 0 r STARTED 1160290 7.2gb 7776371602 127.0.0.1 Jenkins, Abner wiki 1 r INITIALIZING 0 120.3mb 126157581 127.0.0.1 Feline wiki 1 p STARTED 1159509 7.5gb 8116295811 127.0.0.1 Amphibius wiki 1 r STARTED 1159509 7.5gb 8116295811 127.0.0.1 Jenkins, Abner ### Single node filter by index, sort reverse by bytes You can limit the results to a substring match of an index. This filters that output's sixth column through a descending sort. % es shards wik | sort -rnk6 wiki 1 r STARTED 2.7gb 2980767835 276016 127.0.0.1 Namora wiki 0 r STARTED 2.7gb 2953985585 276441 127.0.0.1 Namora wiki 1 p STARTED 2.7gb 2909784771 276016 127.0.0.1 Android Man wiki 0 p STARTED 2.6gb 2846741702 276441 127.0.0.1 Android Man ### Normal three-node cluster operation Add column names. % es shards -v index shard pri/rep state docs size bytes ip node wiki 0 p STARTED 1160290 7.2gb 7776371641 127.0.0.1 Feline wiki 0 r INITIALIZING 0 3.1gb 3384641066 127.0.0.1 Amphibius wiki 0 r STARTED 1160290 7.2gb 7776371602 127.0.0.1 Jenkins, Abner wiki 1 r INITIALIZING 0 3.7gb 4029041251 127.0.0.1 Feline wiki 1 p STARTED 1159509 7.5gb 8116295811 127.0.0.1 Amphibius wiki 1 r STARTED 1159509 7.5gb 8116295811 127.0.0.1 Jenkins, Abner # Contributing es2unix is written in Clojure. You'll need leiningen 2.0+ to build. % make package # License This software is licensed under the Apache 2 license, quoted below. Copyright 2012-2013 ElasticSearch Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.