We have a reporting platform based on Elasticsearch. I was often asked to add fields or modify a mapping of a field. But once the data is indexed in to ES it cannot be modified. So I developed a utility tool which can modify mapping of an Elasticsearch index. The tool also can backup and restore an index. The tool was designed for small indexes never intended for larger indexes.
How it works
1) The utility uses the Search (Scan) API to get 100 documents at once. Make sure the source is enabled.
2) Changes the mapping. Adds/Removes/Delete the fields.
3) Push the documents back to Elasticsearch using bulk API.
If you have dynamic mapping set to strict. Make sure you set the mapping on the new index before you run the tool. The tool doesn’t modify the source index.
Allegiance Inc (Company I work) for agreeing to open source this tool.
Built using Maven. You can specify the Elasticsearch version in the pom file.
mvn clean package
java -jar ESUtility.jar -changeMapping -clusterName ESVM -esHost es1 -field newfield -mappingType type -newFieldType string -newIndex newindex -oldIndex oldindex
- Changes Mapping of an existing Elasticsearch index.
- Add new fields to existing Elasticsearch index. (Add the new mapping to the index and run the tool like changeMapping)
- Removes a field from exisiting Elasticsearch index.
Help : java -jar ESUtility.jar -changeMapping will give you all the available properties
-changeMapping Change Mapping of an index -clusterName <arg> Elasticsearch cluster name -esHost <arg> Elasticsearch host name -field <arg> Field to change the mapping for -mappingType <arg> Mapping Type -newFieldType <arg> New field type -newIndex <arg> Index to restore to -oldIndex <arg> Original Index -removeField <arg> Remove a field (Optional)
The utility reads the entire index in to memory and writes to a file. Beware of the memory. In the next version I will try to write this to multiple files. Help : java -jar ESUtility.jar -backup will give you all the available properties
-backup Backup index to a file -clusterName <arg> Elasticsearch cluster name -esHost <arg> Elasticsearch host name -index <arg> Index to back up -mappingType <arg> Mapping Type
Reads from a file and restores to Elasticsearch index. Help : java -jar ESUtility.jar -restore will give you all the available properties
-restore Restore index from a file -clusterName <arg> Elasticsearch cluster name -esHost <arg> Elasticsearch host name -file <arg> File to restore backup from -index <arg> Index to back up -mappingType <arg> Mapping Type