Presto plugin配置--入门篇

Posted on

presto 支持多种插件，官方支持的列表可以在presto的Github仓库 或者 官方文档 找到。
插件相关的架构图如下：

image.png | left | 747x522

在presto-server目录下，以[connector-name].properties命名，并遵循文档填一些参数，presto启动时就会自动去找这个插件并读取配置来启动：
例如：

1
2
➜  presto-server ls etc/catalog
mongodb.properties  mysql.properties  postgresql.properties

1
2
3
➜  presto-server cat etc/catalog/mongodb.properties
connector.name=mongodb
mongodb.seeds=localhost

启动presto-server可以看到mongo配置文件读取和插件load的相关日志：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
2018-05-18T14:55:08.148+0800    INFO    main    com.facebook.presto.metadata.StaticCatalogStore -- Loading catalog etc/catalog/mongodb.properties --
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       PROPERTY                           DEFAULT       RUNTIME            DESCRIPTION
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.connection-timeout         10000         10000
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.connections-per-host       100           100
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.credentials                []            []
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.cursor-batch-size          0             0
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.implicit-row-field-prefix  _pos          _pos
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.max-wait-time              120000        120000
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.min-connections-per-host   0             0
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.read-preference            PRIMARY       PRIMARY
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.required-replica-set       null          null
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.schema-collection          _schema       _schema
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.seeds                      []            [localhost:27017]
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.socket-keep-alive          false         false
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.socket-timeout             0             0
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.ssl.enabled                false         false
2018-05-18T14:55:08.461+0800    INFO    main    Bootstrap       mongodb.write-concern              ACKNOWLEDGED  ACKNOWLEDGED
2018-05-18T14:55:08.668+0800    INFO    main    org.mongodb.driver.cluster      Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
2018-05-18T14:55:08.705+0800    INFO    main    com.facebook.presto.metadata.StaticCatalogStore -- Added catalog mongodb using connector mongodb --
2018-05-18T14:55:08.706+0800    INFO    main    com.facebook.presto.execution.resourceGroups.InternalResourceGroupManager       -- Loading resource group configuration manager --
2018-05-18T14:55:08.934+0800    INFO    main    Bootstrap       PROPERTY                     DEFAULT  RUNTIME                   DESCRIPTION
2018-05-18T14:55:08.934+0800    INFO    main    Bootstrap       resource-groups.config-file  null     etc/resource_groups.json
2018-05-18T14:55:09.225+0800    INFO    cluster-ClusterId{value='foobar', description='null'}-localhost:27017     org.mongodb.driver.connection   Opened connection [connectionId{localValue:1, serverValue:90242}] to localhost:27017
2018-05-18T14:55:09.287+0800    INFO    cluster-ClusterId{value='foobar', description='null'}-localhost:27017     org.mongodb.driver.cluster      Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 4, 13]}, minWireVersion=0, maxWireVersion=5, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=null, roundTripTimeNanos=60907875, setName='27017', canonicalAddress=localhost:27017, hosts=[localhost:27017, 127.0.0.1:27017], passives=[], arbiters=[127.0.0.1:127017], primary='localhost:27017', tagSet=TagSet{[]}, electionId=7fffweg000000000000001, setVersion=4, lastWriteDate=Fri May 18 14:55:01 CST 2018, lastUpdateTimeNanos=1198867631826184}
2018-05-18T14:55:09.386+0800    INFO    main    io.airlift.bootstrap.LifeCycleManager   Life cycle starting...
2018-05-18T14:55:09.386+0800    INFO    main    io.airlift.bootstrap.LifeCycleManager   Life cycle startup complete. System ready.

使用客户端查看catalogs可以看到有MongoDB:

1
2
3
4
5
6
presto:default> show catalogs;
  Catalog
------------
 mongodb
 ...
(5 rows)

参考文档:
Presto Connector: https://prestodb.io/docs/current/connector