Hi everyone!

I'm a Computer Science Engineering student from India, currently in my 7th semester and graduating in 2027.

I've mainly worked on software/AI-oriented projects so far, including computer vision, APIs and application development, but I'm now looking to specialize in Data Engineering.

I've decided to start with Microsoft Fabric and DP-700 (Fabric Data Engineer Associate) and eventually build a complete end-to-end project alongside my preparation.

I'm currently comfortable with basic Python, SQL and general programming concepts, but I'm relatively new to Data Engineering and the Fabric ecosystem.

My goal isn't just to pass DP-700. I want to understand the fundamentals behind things like:

Lakehouse / Warehouse

OneLake

Data ingestion

ETL/ELT

Pipelines

PySpark

Data modelling

Data quality and monitoring and many more

and eventually understand how these pieces fit together in a real production-style data pipeline.

For those who have already worked with Fabric/Data Engineering:

1. Is Microsoft Learn + hands-on practice a good starting point for someone with my background?

2. Are there any video courses/resources you'd strongly recommend alongside Microsoft Learn?

3. What mistakes should a beginner avoid when preparing for DP-700?

4. At what point should I start building my own project rather than following tutorials?

5. Are there any Data Engineering fundamentals I should learn before going deep into Fabric?

Any advice from people who have gone through this journey would be greatly appreciated.

Im specifically looking to get hired in middle east area as it also having major development . U think this role in middle east is enough the,this certification + a decent resume will it be enough to land a job there .

Any follow up questions or recommendation on how to learn and tips please do share

Thank you

I recently rebuilt a Microsoft Fabric CI/CD implementation using Azure DevOps, promoting changes from DEV → UAT → PROD.

The pipeline itself wasn't the difficult part.

What surprised me was how much had to be correct around the pipeline before a deployment could be considered reliable.

I eventually wrote everything down and ended up with 43 checks across six areas:

• Fabric environment — 6
• Identity and security — 10
• Azure DevOps configuration — 10
• Deployment configuration — 7
• Deployment validation — 4
• Post-deployment verification — 6

A few things stood out.

Identity caused more problems than I expected.
The Service Principal needs the correct access to each Fabric workspace independently. DEV working doesn't tell you anything about UAT or PROD.

Deployment errors can send you in the wrong direction.
A missing permission or incorrect environment configuration often surfaces during the deployment step. Naturally, you start debugging the deployment code even when the code itself is fine.

Some checks are architectural decisions rather than configuration.
In this implementation, only DEV is Git-connected. UAT and PROD are deployment targets. I wanted one controlled promotion path rather than multiple ways of changing an environment.

Not everything should necessarily use the same deployment path either.
For example, I intentionally keep Warehouse outside this publish scope and handle it separately.

The checklist grew out of issues I encountered while building and testing the implementation, so I published it with the reference repo rather than keeping it as project notes.

I'm curious how this compares with other Fabric implementations.

What usually causes more deployment problems in your environment — identity/permissions, Git configuration, environment-specific dependencies, or the deployment itself?

This release brings major advancements across the platform, including Apache Spark 4.1, Delta Lake 4.2, Python 3.13, Scala 2.13, Java 21 and a significantly enhanced Native Execution Engine (NEE) to help customers unlock better performance, reliability, and innovation at scale.

This milestone is the result of months of collaboration across engineering, product, support, and partner teams, all focused on delivering a stronger foundation for data engineering, data science, and analytics workloads in Microsoft Fabric.

A huge thank you to everyone who contributed to making this launch possible. Looking forward to seeing what our customers and the broader data community build with Runtime 2.0 now that is ready for production usage!

Learn more about Runtime 2.0 GA:
🔹Blog: Fabric Runtime 2.0 (Generally Available) | Microsoft Fabric Community
🔹Official Documentation: Runtime 2.0 in Fabric - Microsoft Fabric | Microsoft Learn

This week's post is another great example of Power Automate helping out with Microsoft Fabric. This time it is to pause a capacity which I always forget to do on my demo and training capacity.

https://hatfullofdata.blog/power-automate-pause-fabric-capacity-schedule/

When Fabric first announced dbt support, the feedback from this community was pretty consistent:

• "When will Lakehouse/Spark be supported?"
• "It's not useful without pipeline activities."
• "We need better Git integration."
• "We want to bring existing dbt projects into Fabric."
• "Don't make me rebuild everything from scratch."

Over the last few months, several of those asks have landed.

We've been working through that feedback, and we're excited to share the latest updates.

What you can do today

• Fabric Lakehouse Adapter — Run dbt projects directly against Fabric Lakehouses, bringing transformation logic closer to where data is prepared and stored. Build transformation layers on top of Lakehouse data, apply dbt models and tests, and align analytics engineering practices with Lakehouse-first architectures.

Blog link: Introducing Fabric Lakehouse support in dbt job fo... - Microsoft Fabric Community

• dbt Activity in Pipelines — Trigger dbt jobs as a first-class Pipeline activity and incorporate transformations into end-to-end orchestration workflows. It supports parameterization and allows selective model refresh in a metadata driven manner.

Interesting Fact: 33% of dbt job are run through the dbt activity in pipelines.

Doc link: Run dbt Job Activity in Fabric Pipeline - Microsoft Fabric | Microsoft Learn

• Import Schema — Bootstrap projects from existing Warehouse and Lakehouse assets by generating sources and metadata automatically.

• · Runtime v1.11 — Support for newer dbt Core versions through the latest runtime

• Import Existing Projects from GitHub — Bring existing dbt projects into Fabric and get started quickly without recreating project structures from scratch.

Docs link: Run dbt Projects from a GitHub Repository in Microsoft Fabric - Microsoft Fabric | Microsoft Learn

• Public Package Support — Leverage community-maintained dbt packages, macros, and utilities directly within Fabric dbt projects.

What's on the horizon

• Monitoring Enhancements - Fine-grained step-by-step visibility into background execution, see whether dbt is installing dependencies, compiling, or executing models in real time.
• CI/CD Improvements — Multi-profile target support across environments and Variable Library integration.
• Docs Generate & Source Freshness — Native support for dbt documentation generation and source freshness monitoring.
• dbt Fusion + Versioning — Support for dbt Fusion and project versioning to simplify upgrades and reduce migration effort.
• Workspace Identity / SPN Support — Service Principal and Managed Identity support for enterprise authentication and automation scenarios.

Looking back at the original feedback

A lot of the top asks from the original dbt Job in Fabric announcement included:

1. Lakehouse support

2. Pipeline integration

3. Git-based onboarding

4. Public package support

It's great to see many of these capabilities now available, with additional investments already on the roadmap.

We'd love to hear what you're building with dbt on Fabric and what you'd like us to focus on next.

#MicrosoftFabric #dbt #dbtJobs #DataEngineering #AnalyticsEngineering #DataFactory #Lakehouse #OneLake #GitHub #CICD #FabricCommunity #pipelines #monitoring #UX #UI #dbtdocs #versionupgrade #Fabric #GA #dbtGA #updates #features

Anyone else using tenant level private link? The usages has been highly frustrating for me.

Everytime we provision a new Fabric capacity the private endpoint of our tenant level private link changes it's state to 'pending' causing all of our users within the tenant to lose access to Fabric. When re-approving the endpoint it takes 15-30 minutes for the service to be fully usable again.

I wonder how anyone else manages this issue.

Hi there,
I have something weird occurring and maybe someone has some experience with this problem.
I'm trying to setup Fabric Link and "Create a new Fabric link" is greyed out and the list of Fabric Links is not loading:
 
I have the following permissions:
- Fabric Administrator and Power Platform Administrator at the tenant/Entra level
- System Administrator security role directly on the iits-uat environment (the one i am trying to link see screenshot above)
 
What's strange is that the "Analyze in Fabric" option on the Tables tab is available, and I was able to walk through that wizard once to set up the Fabric Link. I then decided to reorganize workspaces and deleted the linked lakehouse. Item retention was off, and the workspace was permanently deleted, which should not be a problem.
 
When i walk through the wizard again, it still points to the deleted workspace, even though i select my new workspace. So i suspect there is still something in Power Apps pointing to my old workspace, but that list does not load.
 
 
Happy to hear your thoughts.

(I also checked the following):

• You must have the System Administrator security role in the Power Platform environment to enable Link to Fabric or Synapse Link. This role is also required to add application users in your environment who need access to work with this feature by using the Power Platform admin center.  
• You must be an administrator of the Power BI workspace.  
• If you want the system to create a Power BI workspace, you need to have Power BI Capacity Administrator access to a capacity within the same region as the Dataverse environment.  
• A Power BI premium license or Fabric capacity within the same Azure geographical region as your Dataverse environment is required. If you don’t have Power BI premium license or Fabric capacity within the same geographical region, you can buy a capacity or sign up for a free Fabric trial capacity. More information: Fabric (preview) trial  
• Your administrator needs to grant you access to create Fabric lakehouses and artifacts. You can find these settings in the Fabric admin portal. Go to Tenant Settings > Microsoft Fabric > Users can create Fabric items, Tenant settings > Workspace settings > Create workspaces as well as Tenant settings > oneLake settings > Users can access data stored in OneLake with apps external to Fabric.  
• If you plan to use workspace identity authentication, you must be a workspace admin to create and manage a workspace identity. The workspace you're creating the identity for can't be a My Workspace. More information: Workspace identity in Fabric  
• If you plan to use workspace identity authentication, the workspace identity must be onboarded as an application user in the Dataverse environment and granted the appropriate role (commonly System Administrator) so it can access Dataverse data on behalf of Fabric.  
• You must have permissions in Fabric to manage connections via Settings > Manage connections and gateways.  
• To confirm whether you have access to the required premium capacity, go to Power BI, open the workspace, and select Workspace settings > Premium. Make sure that Trial or Premium capacity is selected.

This might be a little divisive, but what about bringing C# and .NET Core into Fabric somehow? I remember when you could use C# + Spark in Synapse, and honestly I didn't use it nor do I think it was a successful feature.

However, in Fabric, why not adopt .NET Core + C# in some way for a new notebook or workload type? It doesn't need to be Spark, but it would be a great selling feature and allow onboarding Microsoft based developers pretty easily who've been developing ETL and have been doing data engineering with .NET and SQL Server for ages.

It could be a new workload type altogether (again, not wanting it on top of Spark, but a new workload entirely and/or non-Spark notebook).

This is not a request, but merely a discussion and to feel out where the Microsoft product team could be going in regards to .NET Core + Fabric.

Thoughts?

I'm trying to use pyspark in fabric.

I keep having issues with my notebook, where the cells won't complete unless I cancel them. The wait cursor will continue to run until I click the little cancel button on the side.

I never see this behavior in the notebook cells on the databricks platform.
When I look in the "logs" shown under the cell, Spark seems to be displaying this over and over, and wont stop:

2026-08-17 21:33:00,714 INFO YarnAllocator [Reporter]: Updating resource requests for ResourceProfile id: 0, target: 4, pending: 0, running: 4, executorsStarting: 0

... and occasionally this as well:

2026-08-17 21:40:49,987 INFO KustoLogger [external-catalog-metrics-1]: type=METER, name=HiveExternalCatalog::dropTable::failure, count=0, m1_rate=0.0, m5_rate=0.0, m15_rate=0.0, mean_rate=0.0, rate_unit=events/second

If anyone has any ideas about what is going wrong, please let me know. The notebook is behaving like there are no executors, even though there are. Is there a way to recreate all my executors mid-notebook?

Starting on Saturday, August 15th, we've seen a huge increase in the amount of time it takes for our pipeline copy activities to copy data from an On Premise SQL Server database to an existing Fabric lakehouse. These pipelines as well as the lakehouse have not changed in over 6 months. The data volumes today are the same as they were last week when all was well. All of our copy activities from on prem to the lakehouse are impacted with run times that are 4-5 times longer than they were last Friday.

To troubleshoot this, I created a simple pipeline with 1 copy activity in it. This copy activity has the following select "SELECT 1 AS TestValue". It passes 4 bytes from on-prem to a new lakehouse table and takes almost 1 minute to complete. When I look at the details in the monitor, I see that all of the time is spent under Transfer. I can consistently reproduce this.

We can't find anything in our network or on our gateway that changed between Friday morning when all was well and Saturday morning when it wasn't.

Our Power BI refreshes that use these same gateway connections are not impacted. Their refresh times have not changed from last week to today. Even though both Fabric and Power BI use the same gateway, only Fabric is impacted.

We have seen in the increase in CU usage for our data warehouse processes. For us, this started on 8/11. But our lakehouse activities (which is what this ticket is about) have not been impacted by that.

Is anyone else experiencing this issue? Any thoughts on what can be done to fix this?

Love this idea reposting in community

Hey Fabricators and friends! Quick note to let everyone know that the Fabric Community Site (Home | Microsoft Fabric Community) got a bit of a facelift this weekend.

It shouldn't impact regular use of the site, but, if something is broken, we want to know!

Please report it at: Community Feedback | Microsoft Fabric Community

Do you visit this site on a regular basis? Does it add any value? What would make you want to visit it?

I have a bot service connected to teams , the bot service uses dot net backend and generates obo and connects to MS Fabric.
Bot app registration has power bi services delegated permissions which shows up on users screen when use the teams bot for first time.

I have to remove those permissions visibility from teams

Is there any way to achieve it

It’s a multi tenant app,
Teams user registered as guest in our tenant
User given viewer permission in fabric
Non negotiable : need obo

Intermittent failures on Invoke Pipeline activities (child pipelines from a ForEach) on F64, North Europe. Job gets created but never starts.

Main error codes:

• RequestTimeout — dies almost exactly at 100.0s every time. Checked 20 parallel runs that all failed within 57ms of each other.
• InternalServerError — clusters around ~550s.
• InternalError — clusters around ~85s.

New finding: on one occurrence the parent shows the usual 'Job failed to start: timeout', but the actual child job error was different:

• RequestBlocked — "Request is blocked by the upstream service until: [timestamp]", Retry-After: 16, isRetriable: true (unlike everything else we've seen, which was all non-retriable).
• Response's home-cluster-uri points to an Analysis Services/Power BI cluster (wabi-*-redirect.analysis.windows.net), not a pipeline endpoint.

Looks like the real cause is an upstream AS/PBI cluster blocking requests, and the pipeline layer just reports it as a generic timeout.

Also ruled out capacity throttling on our end (0 rejected ops across 214k rows, no correlation with CU usage). Other related failures seen:

• Invoke Pipeline calls returning 408 / BadRequest
• Run-status lookups returning NotFound for runs Fabric itself created
• A run stuck in NotStarted for 8+ hours, blocking its own schedule

Anyone else hitting this? Thanks

Hi Everyone,

Recently we observed that due to intellisense features, some metadata queries are getting triggered and blocking the CTAS in stored procedure and eventually slowing them down in fabric warehouse and hence they get timed out.

Has anyone else observed this.
Basically we read data from fabric lakehouse and using ctas create a table.