Skip to content
Toggle navigation
P
Projects
G
Groups
S
Snippets
Help
ipgallery
/
devops
This project
Loading...
Sign in
Toggle navigation
Go to a project
Project
Repository
Registry
Issues
0
Merge Requests
0
Wiki
Settings
Activity
Graph
Charts
Create a new issue
Commits
Issue Boards
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Commit
432e4f3b
authored
Aug 18, 2019
by
Amir Aharon
Browse files
Options
_('Browse Files')
Download
Email Patches
Plain Diff
add samore
parent
e99bacf1
Expand all
Show whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
20 additions
and
1 deletions
common/docs/Accounts.odt
common/docs/elasticsearch-rest.txt
common/scripts/install_protobuf3.sh
products/MCX/External/activity-esmapping.json
common/docs/Accounts.odt
View file @
432e4f3b
No preview for this file type
common/docs/elasticsearch-rest.txt
View file @
432e4f3b
...
...
@@ -9,7 +9,7 @@ curl 172.16.1.72:9200/_cat/indices?v
curl -XPUT "http://localhost:9200/activityidx" -d'{ "mappings": { "activity": { "properties": { "metaData.loc": { "type": "geo_point" } } } } }'
- https://www.elastic.co/blog/changing-mapping-with-zero-downtime
Steps to change mapping type by reindexing and alias on a new index (using the kibana tools):
1 - get the mapping of the original index: GET activityidx/_mapping
1 - get the mapping of the original index: GET activityidx/_mapping
(delete the starting field 'activityidx)
2 - create a json file with the changed mapping
3 - curl -XPUT localhost:9200/activityidx_v1 -H 'Content-Type: application/json' -d @index-mapping (using curl)
4 reindex to the new original to the new one:
...
...
@@ -203,6 +203,8 @@ POST /activityidx/activity/_search
}\
}
{ "query": { "bool": { "must": [ { "query_string":{ "query":"tenant:fremont & type:\"report/traffic\" & metaData.event:TrafficJam" } } ], "filter" : { "geo_bounding_box" : { "metaData.loc" :{ "top_left" : { "lat" : 37.460391, "lon" : -122.167106 }, "bottom_right" : { "lat" : 37.152796, "lon" : -121.538847 } } }}}}}
///CSV
pip install es2csv
...
...
@@ -214,6 +216,7 @@ es2csv -q 'type:"thing/traffic-lane" AND userId:"chicago/*" AND creationTime:>15
es2csv -r -q '{ "query":{"bool":{"must":[{"query_string":{"query":"tenant:fremont & type:\"thing/traffic-lanes\"" }}],"filter":{"geo_bounding_box":{"metaData.loc":{"top_left":{"lat" : 37.629134,"lon" : -122.173405 }, "bottom_right" : { "lat" : 37.457418, "lon" : -121.835116 }}}}}}}' -u 172.16.1.72:9200 -o activities-traffic-fremont.csv -m 100000 -i activityidx -f creationTime id type metaData.loc metaData.toLocation metaData.measures.speed
es2csv -r -q '{ "query": { "bool": { "must": [ { "query_string":{ "query":"metaData.event:TrafficJam" } } ], "filter" : { "geo_bounding_box" : { "metaData.loc" :{ "top_left" : { "lat" : 37.460391, "lon" : -122.167106 }, "bottom_right" : { "lat" : 37.152796, "lon" : -121.538847 } } }}}}}' -u 167.99.206.187:9200 -o activities-sanjose-traffic-reports.csv -m 5000000 -i activityidx -f creationTime id type metaData
// CURATOR
command line app for managing elasticsearch indices.
...
...
common/scripts/install_protobuf3.sh
0 → 100644
View file @
432e4f3b
#! /bin/bash
# Make sure you grab the latest version
curl
-OL
https://github.com/google/protobuf/releases/download/v3.6.1/protoc-3.6.1-linux-x86_64.zip
https://github.com/google/protobuf/releases/download/v3.6.1/protoc-3.6.1-linux-x86_64.zip
# Unzip
unzip protoc-3.6.1-linux-x86_64.zip
-d
protoc3
# Move protoc to /usr/local/bin/
sudo
mv protoc3/bin/
*
/usr/local/bin/
# Move protoc3/include to /usr/local/include/
sudo
mv protoc3/include/
*
/usr/local/include/
# Optional: change owner
sudo
chown
$USER
/usr/local/bin/protoc
sudo
chown
-R
$USER
/usr/local/include/google
products/MCX/External/activity-esmapping.json
0 → 100644
View file @
432e4f3b
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment