Magellan
Magellan makes it easier for your data scientists, machine learning engineers, and analysts to discover data within your organization by providing a nicer UI on top of the AWS Glue Data Catalog.
Status
Magellan is currently just an alpha preview.
Features
Full text search for tables defined in your metastore
Browse tables in your metastore
Schema & Metadata Discovery
Planned Future Features
- Lineage
- OIDC authentication (e.g. Okta)
Why Use Magellan?
Magellan might be a good fit for you if:
-
You’re on AWS
-
You like the deep integration AWS Glue Data Catalog offers with other AWS servies like Athena, and Hive metastore compatable frameworks such as Apache Spark.
-
You’d like to expose AWS Glue’s catalog via a nicer UI / allow employees to authenticate via OIDC without needing AWS credentials.
-
You want to get up and running with a minimal amount of infrastructure (just copy Magellan’ React UI to S3 + deploy a Lambda APi to talk to Glue). And you don’t want to have to touch Hadoop, K8s and/or EKS and deploy a UI, a search service, a metadata ingestion service, and a databasse just to run your data catalog.
Alternatives
Magellan’s goals overlap with other open source offerings such as: Amundsen, Apache Atlas and DataHub.
Generally speaking these other options are more feature rich, but require a lot more infrastructure setup compared to Magellan.
