Extractors
Meteor currently supports metadata extraction on these data sources. To perform
extraction on any of these you need to create a recipe file with instructions as
mentioned here. In the sample-recipe.yaml
add source information such as type from the table below and config for
that particular extractor can be found by visiting the link in the type field.
Extractors Feature Matrix
Table
| Type | Attributes | Profile | Schema | Lineage | Ownership | Custom |
|---|---|---|---|---|---|---|
clickhouse | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
couchdb | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
mongodb | ✅ | ✅ | ✗ | ✗ | ✗ | ✗ |
mssql | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
mysql | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
postgres | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
cassandra | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
oracle | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
mariadb | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
redshift | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
presto | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
snowflake | ✅ | ✅ | ✅ | ✗ | ✗ | ✗ |
Dashboard
| Type | Url | Chart | Lineage | Tags | Custom |
|---|---|---|---|---|---|
grafana | ✅ | ✅ | ✗ | ✗ | ✗ |
metabase | ✅ | ✅ | ✗ | ✗ | ✗ |
superset | ✅ | ✅ | ✅ | ✗ | ✗ |
tableau | ✅ | ✅ | ✅ | ✗ | ✗ |
redash | ✅ | ✗ | ✗ | ✗ | ✗ |
Topic
| Type | Profile | Schema | Ownership | Lineage | Tags | Custom |
|---|---|---|---|---|---|---|
kafka | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
User
| Type | Username | FullName | Title | IsActive | ManagerEmail | Profiles | Memberships | facets | common | |
|---|---|---|---|---|---|---|---|---|---|---|
github | ✅ | ✅ | ✅ | ✗ | ✅ | ✗ | ✗ | ✗ | ✗ | ✗ |
shield | ✅ | ✅ | ✅ | ✗ | ✅ | ✗ | ✗ | ✅ | ✅ | ✗ |
gsuite | ✅ | ✗ | ✅ | ✗ | ✅ | ✅ | ✗ | ✗ | ✗ | ✗ |
Bucket
| Type | Location | StorageType | Blobs | Ownership | Tags | Custom | Timestamps |
|---|---|---|---|---|---|---|---|
gcs | ✅ | ✅ | ✗ | ✅ | ✅ | ✗ | ✅ |
Job
| Type | Ownership | Upstreams | Downstreams | Custom |
|---|---|---|---|---|
optimus | ✅ | ✅ | ✅ | ✅ |
Machine Learning Feature Table
| Type | Ownership | Upstreams | Downstreams | Custom |
|---|---|---|---|---|
caramlstore | ✗ | ✅ | ✗ | ✅ |
Application
| Type | Ownership | Upstreams | Downstreams | Custom |
|---|---|---|---|---|
application_yaml | ✅ | ✅ | ✅ | ✅ |
Machine Learning Model
| Type | Ownership | Upstreams | Downstreams | Custom |
|---|---|---|---|---|
merlin | ✅ | ✅ | ✗ | ✅ |
Generic
These are special type of extractors that are capable of extracting any type of asset.
| Type | Ownership | Upstreams | Downstreams | Custom |
|---|---|---|---|---|
http | ✅ | ✅ | ✅ | ✅ |