Marquez is an open source metadata service for the collection, aggregation, and visualization of a data ecosystem's metadata. It maintains the provenance of how datasets are consumed and produced, provides global visibility into job runtime and frequency of dataset access, centralization of dataset lifecycle management, and much more.
A project of this scope requires input from a wide range of subject matter experts with different backgrounds and allegiances. As such we need a set of principles, roles and operating practices to ensure the results of our contributions are useful, have high quality and are widely consumable.
The principles set the tone of the operation of Marquez:
- The activities of the project ensure open collaboration. Through this open collaboration we aim to build a community of people who are interested in the success of the project.
- The scope of the content is determined by the individuals who are actively contributing.
- The resulting content is licensed under the Apache 2.0 license.
- An individual's privileges and position is awarded through their contribution and engagement.
These principles should be respected as the procedures used to manage the Marquez project are evolved and matured.
Anyone can become a member of the Marquez community by signing up to the the Marquez mailing list, joining the Slack community, attending the virtual community meetings or contributing content to one or more of the GitHub repositories.
The Contribution Guide describes how to connect to these channels.
All participants in the Marquez community are bound by the project's Code of Conduct.
As a member you are able to attend our meetings, just to listen, or to play an active part in the discussion. The online meetings are recorded to allow community members to catch up if they are not able to attend the live meeting. When you attend the community meetings specifically, your name will be recorded in the meeting minutes along with any remarks or suggestions you make. The agenda and minutes of our community meetings are publicly available on the wiki.
A member may make contributions to the Marquez content by submitting a GitHub pull request on the appropriate Git repository. This will be reviewed and processed by the Marquez committers.
Each contribution is signed by the contributor to confirm they agree to our Developer Certificate of Origin (DCO).
Community members can progress to be Marquez Contributors and then Marquez Committers.
Marquez contributors are members who have actively taken additional steps to promote and foster the success of Marquez and its acceptance/adoption across the IT community. The activities that contributors engage in might include:
- Provide best practices for information governance, lineage, metadata management and other related disciplines during active discussions and/or development of material
- Actively participate in meetings and discussions
- Promote the goals of Marquez and the benefits of open metadata to the IT community (deliver presentations, informal talks, assist at trade shows, independent blogs, etc.)
- Assist in the recruitment of new members
- Contribute where appropriate to documentation and code reviews, specification development, demonstration assets and other artifacts that help move Marquez forward
Committers are members of the Marquez community who have permission to change the Marquez content. This may be content that they have created themselves, or has been provided by another member. Committers also have responsibility for helping other project members with their contributions. This includes:
- Monitoring email aliases.
- Monitoring Slack (delayed response is perfectly acceptable).
- Triaging GitHub issues and performing pull request reviews for other maintainers and the community.
- Making sure that ongoing Git pull requests are moving forward at the right pace or closing them.
New committers are voted onto the committers list by the existing committers. For the current list, see: COMMITTERS.md.
The committers vote, and if a majority agree then the requester is added to the committers list and given write access to the Git repositories.
Once confirmed, you can publicly refer to yourself as a Marquez committer.
If a committer is no longer interested or cannot perform the committer duties listed above, they should volunteer to be moved to emeritus status. In extreme cases this can also occur by a vote of the committers per the voting process below. Emeritus committers can rejoin the committer list through a vote of the existing committers.
The leadership of Marquez is granted through a vote of the Marquez committers. Marquez is currently led by Willy Lulciuc.
Some meetings are face-to-face, but most are conference calls. Attendance at meetings is open to all. Conference calls can be joined without an explicit invitation. However, due to physical security requirements at some of the venues we use, it is necessary to ensure you are added to the invitee list of any face-to-face meetings that you wish to attend and complete the necessary formalities for the venue.
For example, the face-to-face meeting may be at a conference that requires you to register for the conference to attend. Or a meeting may be at an organization's offices that are required to maintain a list of everyone on site.
Marquez uses a Slack community to provide an ongoing dialogue between members. This creates a recorded discussion of design decisions and discussions that complement the project meetings.
Follow the link above and register with the Slack service using your email address. Once signed in you can see all of the active Slack channels.
Additional channels are added from time to time as new workgroups and discussion topics are established.
Marquez uses a discussion list for more general discussion.
The Marquez content is managed in GitHub under https://github.com/MarquezProject/marquez. It may be developed using patches, branches from main, or forks/git pull requests. Each change should have a GitHub issue explaining why the change is being made. See: CONTRIBUTING.md
When new content proposed to the project, the person contributing is required to sign the contribution to confirm it conforms to the Developer Certificate of Origin (DCO).
The Marquez team aim to create regular official release of Marquez.
In between official releases, the latest build is also available to developers on GitHub.
For details about the timing and authorization of releases, see: RELEASING.md.
In general, we prefer that technical issues and committer membership are amicably worked out between the persons involved. If a dispute cannot be decided independently, the committers can be called in to decide an issue. If the committers themselves cannot decide an issue, the issue will be resolved by voting. The voting process is a simple majority in which each committer receives one vote.
SPDX-License-Identifier: Apache-2.0 Copyright 2018-2023 contributors to the Marquez project.