This was very interesting news for me. Microsoft started a project for metadata management within the Business Intelligence area. We have seen already a first solution within SQL Server Denali in the form of data lineage analysis for SQL Server Integration Services. I don’t know why they started now a complete new project with a new architecture but I hope we will get some information soon. It is also not clear when and how this will shipped.
But let’s get into some more details. What is it about? The next picture shows an overview of the basic architecture.
There will be a couple of crawlers for SQL, SSIS, SSR, SSAS, SharePoint, Excel, … (also 3rd party) that will be able to extract metadata and dataflow information from those sources that are indexed with in Barcelona Index Server. It was also said, that for sources where you can’t extract metadata you will be able to describe them.
The Index Server will be the cache for all of the metadata and expose the information via the Metadata Graph Query and Augmentation API to different clients. I like the idea of having an API which we see in nearly all of the Microsoft products now.
There will be clients for administration, DBAs, Information Worker and there is also space for 3rd party software.
What can we do with this metadata? I‘m very curious how the UIs will look like and how the metadata will be exposed (hopefully also in a nice graphical way) but there are some common scenarios, e.g.
- Where does my data originally come from that I have selected in my reports?
- Which SSIS packages / Cubes / Reports are affected if I change this relational structure?
- How was my data transformed all they way from the source to the report?
If you are interested in this topic I can recommend the Barcelona Team blog: http://blogs.msdn.com/b/project_barcelona_team_blog/
As soon as I got some more details, I’ll let you know.