ES 常见 curl 命令

安装ES后, 不需要输入密码, 就可以直接http端口通过api创建index, 查询和输入数据.

默认安装几乎就是完全暴露在外, 难怪有那么多的ES泄漏事故.

查询 es 版本

rpm安装的版本

curl -X GET "172.16.48.143:9200" -u admin:password
{
  "name" : "domain-172-16-48-143",
  "cluster_name" : "7ef58995-995d-40c7-874e-07c094c49fbc",
  "cluster_uuid" : "eY2DeYBXR_2HJnBQ5vKVYw",
  "version" : {
    "number" : "7.10.2-SNAPSHOT",
    "build_flavor" : "oss",
    "build_type" : "rpm",
    "build_hash" : "fcd0559d93f54209e9259220fa53d3fe54f47709",
    "build_date" : "2023-05-31T18:10:38.566521Z",
    "build_snapshot" : true,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

docker 安装的版本

curl -X GET "es.domain.com:15009" -u elasticsearch:Password
{
  "name" : "es-domain-domain-es-ss-1",
  "cluster_name" : "domain-domain-es",
  "cluster_uuid" : "xBk2htVPToSQ8TSKSIIaHA",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "docker",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

查询 es index 列表

如果es设置了密码, 直接访问会报错没有权限

-bash-4.2# curl -X GET "es.domain.com:15009/_cat/indices?v"
Unauthorized

在curl命令行里支持输入账号密码信息

-bash-4.2# curl -X GET "es.domain.com:15009/_cat/indices?v" -u admin:password


health status index                     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   es_tbl_index_202305312223 S3mAxuLqQP6SMqo-h03aeQ  10   1          6            1    204.4kb         95.6kb
green  open   example_code_search        uiS1wRVBQb2V4X8smXkNpA   3   1          0            2      4.4kb          3.8kb
green  open   .opendistro_security      TcMHF3_iRPy6eUNM-7RJjw   1   2          9            1    167.6kb         59.4kb
green  open   .kibana_1                 V8hMyMDLS2ugIb0LtwaIdQ   1   1          0            0       416b           208b
-bash-4.2# 

创建es index

https://www.elastic.co/guide/cn/elasticsearch/guide/current/_creating_an_index.html

curl --location --request PUT '140.143.218.238:9200/es_test'

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "es_test"
}

查询es index信息

curl --location --request GET '140.143.218.238:9200/es_test'

{
    "es_test": {
        "aliases": {},
        "mappings": {},
        "settings": {
            "index": {
                "creation_date": "1678093946551",
                "number_of_shards": "1",
                "number_of_replicas": "1",
                "uuid": "FJgvtOV8Tg2VxisPuxwD8A",
                "version": {
                    "created": "7060299"
                },
                "provided_name": "es_test"
            }
        }
    }
}

查询不存在的index

curl --location --request GET '140.143.218.238:9200/es_sink'

{
    "error": {
        "root_cause": [
            {
                "type": "index_not_found_exception",
                "reason": "no such index [es_sink]",
                "resource.type": "index_or_alias",
                "resource.id": "es_sink",
                "index_uuid": "_na_",
                "index": "es_sink"
            }
        ],
        "type": "index_not_found_exception",
        "reason": "no such index [es_sink]",
        "resource.type": "index_or_alias",
        "resource.id": "es_sink",
        "index_uuid": "_na_",
        "index": "es_sink"
    },
    "status": 404
}

插入es数据

直接在url里输入index/type/id, 然后es会自动创建对应的type和id.

curl --location --request GET '140.143.218.238:9200/es_test/database/abc' \
--header 'Content-Type: application/json' \
--data-raw '{
    "a": 123,
    "b": "22",
    "c": 23
}'

{
    "_index": "es_test",
    "_type": "database",
    "_id": "abc",
    "_version": 1,
    "result": "created",
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "_seq_no": 0,
    "_primary_term": 1
}

查询es数据

全量查询

curl -X GET "es.domain.com:15009/ds-audit-operation-log/_search?pretty&size=10" --useruser:password 

咨询chatgpt 而来

curl -X GET "es.domain.com:15009/ds-audit-operation-log/_search?pretty&size=3" --user user:password

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 4,
    "successful" : 4,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 5,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "ds-audit-operation-log-v1",
        "_type" : "_doc",
        "_id" : "vSnumYgBKEFNM_IYAsjD",
        "_score" : 1.0,
        "_source" : {
          "TenantId" : "1255000001",
          "ProjectId" : null,
          "ClusterId" : null,
          "ModuleName" : null,
          "OperationTime" : "2023-06-08 15:34:52",
          "OperationUserId" : "100500000018",
          "OperationUserName" : null,
          "OperationObject" : "PROJECT",
          "OperationType" : "MODIFY_DATA_SOURCE",
          "OperationStatus" : "SUCCESS",
          "Details" : "\"{\\\"url\\\":\\\"jdbc:mysql://58.87.83.245:59679/domainDB\\\",\\\"port\\\":\\\"59679\\\",\\\"vip\\\":\\\"58.87.83.245\\\",\\\"vurl\\\":\\\"jdbc:mysql://58.87.83.245:59679/domainDB\\\",\\\"username\\\":\\\"domain\\\",\\\"connectType\\\":\\\"public\\\",\\\"authorityType\\\":\\\"true\\\",\\\"ip\\\":\\\"58.87.83.245\\\",\\\"type\\\":\\\"MYSQL\\\",\\\"deployType\\\":\\\"CONNSTR_PUBLICDB\\\",\\\"vpcTenantId\\\":\\\"1255000001\\\",\\\"superSQLBand\\\":\\\"false\\\",\\\"password\\\":\\\"****\\\",\\\"vport\\\":\\\"59679\\\",\\\"db\\\":\\\"domainDB\\\"}\""
        }
      },
      {
        "_index" : "ds-audit-operation-log-v1",
        "_type" : "_doc",
        "_id" : "vikAmogBKEFNM_IY9sh_",
        "_score" : 1.0,
        "_source" : {
          "TenantId" : "1255000001",
          "ProjectId" : "1729096905026240512",
          "ClusterId" : null,
          "ModuleName" : "CONSOLE",
          "OperationTime" : "2023-06-08 15:55:30",
          "OperationUserId" : "100500000065",
          "OperationUserName" : "domain",
          "OperationObject" : "PROJECT",
          "OperationType" : "ADD_MEMBER",
          "OperationStatus" : "SUCCESS",
          "Details" : "{\"userName\":[\"domain\"],\"userRole\":[\"项目管理员\"]}"
        }
      },
      {
        "_index" : "ds-audit-operation-log-v1",
        "_type" : "_doc",
        "_id" : "OHgQmogBdy1dKbofvtmc",
        "_score" : 1.0,
        "_source" : {
          "TenantId" : "1255000001",
          "ProjectId" : "1729096905026240512",
          "ClusterId" : null,
          "ModuleName" : null,
          "OperationTime" : "2023-06-08 16:12:52",
          "OperationUserId" : "100500000020",
          "OperationUserName" : null,
          "OperationObject" : "PROJECT",
          "OperationType" : "CREATE_DATA_SOURCE",
          "OperationStatus" : "SUCCESS",
          "Details" : "\"{\\\"url\\\":\\\"jdbc:mysql://58.87.83.245:59679/domainDB\\\",\\\"port\\\":\\\"59679\\\",\\\"vip\\\":\\\"58.87.83.245\\\",\\\"vurl\\\":\\\"jdbc:mysql://58.87.83.245:59679/domainDB\\\",\\\"username\\\":\\\"aa\\\",\\\"connectType\\\":\\\"public\\\",\\\"authorityType\\\":\\\"true\\\",\\\"ip\\\":\\\"58.87.83.245\\\",\\\"type\\\":\\\"MYSQL\\\",\\\"deployType\\\":\\\"CONNSTR_PUBLICDB\\\",\\\"vpcTenantId\\\":\\\"1255000001\\\",\\\"superSQLBand\\\":\\\"true\\\",\\\"password\\\":\\\"****\\\",\\\"vport\\\":\\\"59679\\\",\\\"db\\\":\\\"domainDB\\\"}\""
        }
      }
    ]
  }
}

url直接定位 document id进行查询

注意这里在url里, 直接指明了index, type, id.

curl --location --request GET '140.143.218.238:9200/es_test/database/abc' \
--data-raw ''

{
    "_index": "es_test",
    "_type": "database",
    "_id": "abc",
    "_version": 1,
    "_seq_no": 0,
    "_primary_term": 1,
    "found": true,
    "_source": {
        "a": 123,
        "b": "22",
        "c": 23
    }
}

es查询

基本上都是咨询chatgpt而来

采样查询几条记录

es 查询前几条记录用于测试

curl -X GET '172.16.16.122:9200/ranger_audits/_search?pretty' -H 'Content-Type: application/json' -d '{
  "size": 5,
  "query": {
    "match_all": {}
  }
}' --user admin:fakePassword

采样查询匹配某个keyword参数的几条记录

curl -X GET '172.16.16.122:9200/ranger_audits/_search?pretty' -H 'Content-Type: application/json' -d '{
  "size": 5,
  "query": {
    "match": {
       "resType": "column"
    }
  }
}' --user admin:fakePassword

查询某个keyword参数的枚举项

curl -X POST '172.16.16.122:9200/ranger_audits/_search?pretty' -H 'Content-Type: application/json' -d '{
  "size": 0,
  "aggs": {
    "result_values": {
      "terms": {
        "field": "resType",
        "size": 10
      }
    }
  }
}' --user admin:fakePassword

返回的结果大概是这样

{
  "took" : 659,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "result_values" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "path",
          "doc_count" : 797159
        },
        {
          "key" : "column",
          "doc_count" : 98529
        },
        {
          "key" : "table",
          "doc_count" : 91192
        },
        {
          "key" : "@database",
          "doc_count" : 72180
        },
        {
          "key" : "@table",
          "doc_count" : 3102
        },
        {
          "key" : "@column",
          "doc_count" : 940
        },
        {
          "key" : "queue",
          "doc_count" : 431
        },
        {
          "key" : "@null",
          "doc_count" : 62
        },
        {
          "key" : "column-family",
          "doc_count" : 22
        },
        {
          "key" : "@udf",
          "doc_count" : 2
        }
      ]
    }
  }
}