Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
1 answer
How to UPSERT ODBC dataset from .csv file?
I need to be able to UPSERT records in an ODBC dataset. The ODBC dataset links to a table in a DB2 database. We receive a .csv file, which is cleaned up and processed through a pipeline and data flow, and the records in the .csv file are what is used…
Azure Data Factory
asked 2025-12-08T15:14:14.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 46%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
AE90
85
Reputation points
commented 2025-12-12T07:46:32.8133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 18%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVP%3C/text%3E%3C/svg%3E)
VRISHABHANATH PATIL
2,305
Reputation points Microsoft External Staff
Moderator
2 answers
for api changes, what if we move the trigger pipeline code to shared?
for api changes, what happens if we move the trigger adf pipeline code to shared? now it is in feature in cqrs architecture
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-02T04:22:24.2466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 99%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Ashwini Kanjapur
0
Reputation points
commented 2025-12-12T06:22:17.2033333+00:00
VRISHABHANATH PATIL
2,305
Reputation points Microsoft External Staff
Moderator
2 answers
ADF Amazon Redshift linked service produces SSL error
I have a Amazon Redshift v1.0 linked service in Azure Data Factory that connects successfully, when I test the connection. When the service is used in a dataset, it fails to fetch data with 27809 error code: Failed while reading data from source during…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-11-20T12:15:04.1133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 92%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVG%3C/text%3E%3C/svg%3E)
Vincents Goldmanis
20
Reputation points
commented 2025-12-12T06:17:30.86+00:00
VRISHABHANATH PATIL
2,305
Reputation points Microsoft External Staff
Moderator
1 answer
need help with Action required for your Data Factory service in Global.
Service: Data Factory Region: Global Event tags: -- You’re receiving this notice because you use Azure Data Factory. You're receiving this notification because you're associated with one or more Azure subscriptions that use Azure Data Factory connectors…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-12T01:15:04.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 97%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
Abhilash Nair (HCL) (Admin)
0
Reputation points
answered 2025-12-12T05:46:58.22+00:00
VRISHABHANATH PATIL
2,305
Reputation points Microsoft External Staff
Moderator
1 answer
Parametrize authentication method of linked services
Hello, we have the following set up: We have three data factory instances dev, preview and prod. Dev is connected to our repository. Preview and Prod are not. For all three instances, we have a linked service X which uses a system assigned identity (SAI)…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-11T16:18:42.3166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 12%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELL%3C/text%3E%3C/svg%3E)
Lina Lampel
0
Reputation points Microsoft Employee
edited the question 2025-12-12T03:17:04.51+00:00
VRISHABHANATH PATIL
2,305
Reputation points Microsoft External Staff
Moderator
2 answers
Slow ADF Copy Performance When Transferring Oracle Tables Containing CLOB Columns
I am facing a performance issue in ADF when copying data from an Oracle source table that contains a CLOB column. Although the table size is only around 1 GB, the copy operation takes almost 2 hours to complete. However, if I convert the CLOB column to a…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-03T11:13:59.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 26%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVT%3C/text%3E%3C/svg%3E)
Vaibhav Tiwari
0
Reputation points
commented 2025-12-11T18:46:07.8866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 24%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
Pilladi Padma Sai Manisha
575
Reputation points Microsoft External Staff
Moderator
3 answers
Unable to download linked ARM templates from private Storage Account in CI/CD deployment
Issue Description: We are deploying ARM templates using Azure DevOps pipelines. For linked templates, we have stored them in an Azure Storage Account (Blob), referenced by URL with a SAS token in our main ARM template. When the storage container is…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-11-18T07:41:28.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 32%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETP%3C/text%3E%3C/svg%3E)
Tej Prakash
5
Reputation points
commented 2025-12-11T16:23:02.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 2%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMK%3C/text%3E%3C/svg%3E)
Manoj Kumar Boyini
1,660
Reputation points Microsoft External Staff
Moderator
2 answers
Files Stored in respective folder
Hello Team, There are multiple files stored in 3 folder in SFTP Location.The pipeline is schduled for every 2 hours. My queries is if any news files is not present on the schdule time then the pipeline must not run. If any new files are present in the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-03T11:38:00.4533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 13%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rohit Kulkarni
731
Reputation points
commented 2025-12-11T16:20:39.4066667+00:00
Rohit Kulkarni
731
Reputation points
1 answer
Copy Activity with maximum N files in 1 trigger?
How can I configure my pipeline/copy activity to copy a maximum of 100 files at a time (one trigger) from one blob storage to another in binary mode (using the delete after copy option), given that the source contains thousands of files?
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2024-08-13T16:10:30.12+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 16%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHP%3C/text%3E%3C/svg%3E)
Hrithik Purwar
25
Reputation points
commented 2025-12-11T14:55:16.9033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 70%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBR%3C/text%3E%3C/svg%3E)
Baker, Rick
0
Reputation points
2 answers
Not able to process the blob data
Hi, We are extracting data from an on-premises DB2 LUW database and loading it into an Azure Database for PostgreSQL. The source tables contain BLOB data types used to store PDF files, and the data copy process is getting stuck on tables with BLOB…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 37%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
2 answers
Authentication type is null in oracle linked service.
Hi, I created the Oracle linked service in my feature branch and merged it into the main branch. The Git pipeline then published these changes to Data Factory live mode. The Oracle linked service was deployed to live mode, but the authentication type is…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
6 answers
One of the answers was accepted by the question author.
Unable to download Purview exports
Hello I have tried to download the export packages from the ediscovery menu. I have tried on mac & linux Edge and firefox browsers. When I push the download button, nothing happens. The close button does work. Is there another way to access these…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-03-03T07:21:08.6833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 10%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECL%3C/text%3E%3C/svg%3E)
Christopher Lowden
195
Reputation points
commented 2025-12-11T12:27:33.4+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 38%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERB%3C/text%3E%3C/svg%3E)
Rob Black
0
Reputation points
1 answer
One of the answers was accepted by the question author.
ADF degree of parallelism
hello sir, I have 10 tables to fetch data from, and I have lookup and foreach activity to iterate through each table with single copy activity and set degree of parallelism to 5 to concurrent data process. I have also set foreach batch count to 15. How…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-09T16:38:17.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 86%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EI%3C/text%3E%3C/svg%3E)
Inquisitive
60
Reputation points
accepted 2025-12-11T10:03:40.48+00:00
Inquisitive
60
Reputation points
1 answer
Slow Data Factory copy data runs
Data Factory copy data runs are sometimes quick to complete with a high throughput ~ 30 mb/s but the same pipeline subsequent times will run very slow <500 kb/s. I am running a SQL script that takes data from an Azure SQL DB transforms it and places…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-09T16:16:51.9266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 72%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERW%3C/text%3E%3C/svg%3E)
Ryan Wu
0
Reputation points
commented 2025-12-10T17:21:03.19+00:00
Manoj Kumar Boyini
1,660
Reputation points Microsoft External Staff
Moderator
1 answer
Hot Iterative Read Operations : Definition and how to manage them?
Posted this question on SO but with no answers so far, so I try it here hoping more luck. We are trying to identify the operations behind the item 'Hot Iterative Read Operations' in our Azure Data Lake Gen2 containers as this seems to be our main cost…
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,585 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2023-10-17T15:46:58.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 38%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
javier cohenar
211
Reputation points
commented 2025-12-10T13:19:13.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 3%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAF%3C/text%3E%3C/svg%3E)
Andrew Freeman
0
Reputation points
2 answers
I want to write a logic to trigger adf pipeline in cqrs architecture so where can i write in shared folder or feature folder
I want to write a logic to trigger adf pipeline in cqrs architecture so where can i write in shared folder or feature folder
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-02T04:27:22.9233333+00:00
Ashwini Kanjapur
0
Reputation points
commented 2025-12-10T04:13:15.74+00:00
VRISHABHANATH PATIL
2,305
Reputation points Microsoft External Staff
Moderator
3 answers
In an high availability configuration of Azure Data Factory, how can I reduce the 5-minute failover/recovery time for nodes that fail during pipeline runs?
I am developing enterprise-grade pipelines that ingest from on-premises data sources using a self-hosted integration runtime and am utilizing a multi-node high-availability configuration in Azure Data Factory. During runs, I've noticed that if a node…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-01T00:23:14.81+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 97%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAW%3C/text%3E%3C/svg%3E)
Alex WB
0
Reputation points
commented 2025-12-10T03:21:56.4733333+00:00
VRISHABHANATH PATIL
2,305
Reputation points Microsoft External Staff
Moderator
1 answer
Microsoft Data Engineer Associate Certication renewal
Certication renewal communication not received hence it gotexpired on 27th nov. can you plshelp to enable for reassessment
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-12-10T02:21:55.54+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 26%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHB%3C/text%3E%3C/svg%3E)
Hariharan Balasubramanian
0
Reputation points
answered 2025-12-10T02:23:00.9333333+00:00
Q&A Assist
3 answers
Email Notifcation in ADF Pipline
Hello Team, I am using the Web activity to received the email notification in my outlook.So i have mentioned this formula @json(concat( '{', '"message": "ERP-DW-UAT TRANSACTION_PARTITION INPROGRESS",', …
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2025-11-13T18:41:39.97+00:00
Rohit Kulkarni
731
Reputation points
commented 2025-12-09T19:54:16.33+00:00
Pilladi Padma Sai Manisha
575
Reputation points Microsoft External Staff
Moderator
2 answers
One of the answers was accepted by the question author.
How to fix the problem : 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete
I am getting this error while using the sink option in data factory The folder I am trying in path is from storage account.
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,585 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
3,291 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,992 questions
asked 2024-08-02T18:58:13.3733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 47%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKT%3C/text%3E%3C/svg%3E)
Karan Tuteja
45
Reputation points
answered 2025-12-09T10:09:13.55+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 19%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAO%3C/text%3E%3C/svg%3E)
Alejandro OP(Office 365 Admin)
0
Reputation points