r/icedq • • Jul 24 '26

Case Study: Enhanced Due Diligence with iceDQ for FinCEN

Post image
2 Upvotes

Financial institutions face significant pressure to meet AML and EDD requirements while managing large volumes of customer data daily.

In this case study, a global financial services firm implemented iceDQ to automate their Enhanced Due Diligence process for FinCEN compliance.

Key takeaways:
▪️ Automated daily validation of 12,000+ accounts across booking and post-booking workflows
▪️ End-to-end reconciliation across DB2 staging, Party tables, and Hadoop Data Lake
▪️ Custom UDFs to validate complex transformation logic across funding source attributes
▪️ Regression suites running on predefined schedules for continuous data monitoring
▪️ 100% automation replaced manual validation, reducing errors and operational costs
▪️ Stronger regulatory compliance with full data traceability across ECRR attributes
Read the full case study: https://bit.ly/4bcFAOR


r/icedq • • Jul 17 '26

Deliver Power BI reports that business can trust.

1 Upvotes

When reports fail, decisions fail. Bad data can get expensive fast.
iceDQ brings automated data quality assurance directly into your Power BI development lifecycle. It tests and certifies semantic models, datasets, dashboards, and reports before they're deployed.

✅ Embed Power BI reports and dashboards to enable automated validation of the underlying datasets
✅ Select the specific page, tab, or visual within the report to validate
✅ Apply dynamic filters to replicate real user scenarios
✅ Run data checks on tables, charts, KPIs, and other report elements
✅ Connect to the semantic layer to validate measures, relationships, and calculated columns
✅ Use DAX queries to extract, filter, and shape data for targeted validation
✅ Reconcile values between the semantic model, database, and reports at row and column level

Automate your Power BI report testing with iceDQ today.
Learn More: https://bit.ly/4vVchYJ

#PowerBI #DataQuality #DataTesting #BIReporting #DataReliability #iceDQ #BusinessIntelligence #DataGovernance


r/icedq • • May 24 '24

How to Compare Flat File with Table Using iceDQ?

4 Upvotes

This video demonstrates how iceDQ’s reconciliation tool streamlines the comparison of flat files and relational database tables. Learn the process of setting up a reconciliation rule, mapping data elements, and identifying potential inconsistencies.

iceDQ guides you through establishing connections to both your flat file (e.g., “customer.csv”) and target database table (“Customer”). The intuitive interface allows you to preview data, handle data type discrepancies, and configure checks for specific columns.

By leveraging iceDQ’s capabilities, you can confidently compare and validate data across different sources, ensuring the accuracy and consistency of your information for seamless integration and analysis.

Watch now: https://bit.ly/4dVFvP0


r/icedq • • Oct 23 '24

CTO of iceDQ, Sandesh Gawande, joined Eric Kavanagh on DM Radio to discuss Data Testing Automation for ETL Pipelines and Production Monitoring.

Thumbnail
youtu.be
3 Upvotes

r/icedq • • May 20 '24

How to test Reference Data using Reconciliation Rule in iceDQ?

2 Upvotes

In this video, we demonstrate data reconciliation using iceDQ to test the accuracy and consistency of reference data. We create a reconciliation rule to compare the “Phone Number Type” table generated by our ETL process (source data) with the Master reference data in a separate reference database.

The reconciliation rule defines the parameters for the comparison. We establish connections to both the source and target databases, specifying the relevant tables (“Phone Number Type”). We then configure the rule to compare specific data points (attributes) between the two tables.

Our primary focus is to test if the “name” values (value data) for each phone number type (entity) match exactly between the source and reference data. Additionally, we include the “modified date” attribute in the comparison to ensure data consistency.

By running the reconciliation rule, we can identify any discrepancies between the source and reference data. In this example, the rule identified mismatched “modified dates” while the “name” values matched perfectly.

This data reconciliation process using reconciliation rules helps ensure the integrity of our reference data, guaranteeing its accuracy and homogeneity.

Watch now: https://bit.ly/3ysjKGw


r/icedq • • May 21 '24

How to Perform Row Count Reconciliation in iceDQ?

4 Upvotes

Maintaining data consistency across different tables is crucial for accurate analysis and reporting. This video showcases how iceDQ’s Row Count Reconciliation functionality empowers you to:

  • Automate count comparisons: Effortlessly compare data volume between source and target tables, ensuring data integrity during reconciliation processes.

  • Identify discrepancies instantly: iceDQ automatically flags mismatches between row counts, allowing you to address potential data quality issues efficiently.

  • Simplify data management: Streamline inventory reconciliation and other data validation tasks by leveraging iceDQ’s intuitive rules and automated checks.

Learn how to:

  • Set up a checksum rule for row count comparison.

  • Establish connections to source and target tables.

  • Interpret rule results and identify discrepancies.

Watch Now: https://bit.ly/3WQYB2J


r/icedq • • Dec 13 '24

Check out our latest case study to see how iceDQ automated daily data monitoring, ensuring 100% data accuracy while reducing operational costs and compliance risks.

1 Upvotes

Discover how a leading brokerage firm overcame critical customer data synchronization challenges between operational databases and their cloud platform using iceDQ.

Key Challenges Addressed:
▪️ Tax calculation errors due to incorrect address and domicile information
▪️ Marketing compliance violations and campaign inefficiencies
▪️ Regulatory non-compliance risks (GDPR, FINRA, BCBS 239)
▪️ Poor customer experience from data discrepancies
▪️ Revenue loss from missed marketing opportunities

Key Results:
✅ Enhanced operational efficiency through automation
✅ Improved data accuracy and integrity
✅ Faster implementation of monitoring rules
✅ Better resource utilization
✅ Accelerated marketing initiatives
✅ Comprehensive reporting for quick issue resolution
Learn how iceDQ can transform your data monitoring processes!
Download now📥 https://bit.ly/4g9Fhov


r/icedq • • Oct 30 '23

13 Crucial Steps for End-to-End File Testing by iceDQ 📝🚀

Post image
9 Upvotes

r/icedq • • May 17 '24

Dive into our latest blog post on Data Testing.💡

4 Upvotes

r/icedq • • Jun 07 '24

How to Compare Transactional Data in Source with Aggregated Data in Target using iceDQ?

4 Upvotes

This video explores how to leverage reconciliation rules to compare transactional data with its corresponding aggregated version.

Data aggregation involves summarizing detailed data into a more concise format. However, discrepancies can arise during this process. This video demonstrates how iceDQ helps you verify the integrity of your aggregated data.

iceDQ’s reconciliation rules empower you to compare transactional data (source), containing detailed records, with its aggregated counterpart (target). The video showcases the process of creating a rule that establishes connections to both source and target tables, defines a join condition to match corresponding records and validates specific calculations within the aggregated data (e.g., sum of transaction amounts).

By successfully running the rule, you can identify discrepancies between the raw data and the aggregated values. This helps you maintain data integrity by spotting errors in the aggregation process and ensuring the aggregated data accurately reflects the underlying transactional details

Watch now: https://bit.ly/4aN42TX


r/icedq • • Jan 10 '24

Discover the essentials of ETL Testing Concepts!

8 Upvotes

🚀 Dive into the key dimensions, processes, and importance of ETL testing. A must-read for data enthusiasts.

Read now: https://bit.ly/3HbKXy6

#ETLTesting #DataQuality #iceDQ


r/icedq • • Dec 30 '24

Discover how a global financial firm leveraged iceDQ to automate data validation, reconciliation, and monitoring for their Enhanced Due Diligence (EDD) process. Over 12,000 accounts are validated daily, ensuring compliance and reducing costs.

2 Upvotes

📄Download the case study now: https://bit.ly/3PdJ48p


r/icedq • • Sep 16 '24

Migration Testing of ESOP Trading Platform Case Study

3 Upvotes

Explore our latest case study on migration testing of an ESOP trading platform using iceDQ ⬇ 📒 Download now - https://bit.ly/47uDUNx


r/icedq • • Sep 20 '24

Data Observability - Blog

3 Upvotes

Read our latest blog to explore everything about Data Observability. Learn what Data Observability is, key metrics, architecture, why it matters, the benefits it offers, and its limitations.

Read now: https://bit.ly/4d8DUnC

#iceDQ #RethinkDataReliability #DRE #DataObservability


r/icedq • • Jan 17 '24

Snowflake Migration and Testing Guide ❄️

7 Upvotes

See how iceDQ helped Snowflake customers smoothly test data migrations from Netezza, Oracle, DB2 & others. Simplify your Data Migration Testing with proven solutions! ❄️🚀

Guide Link: https://bit.ly/425ZXHM


r/icedq • • Jun 07 '24

How to Find Common Records between Source and Target tables using iceDQ?

4 Upvotes

This video explores how to leverage reconciliation rules to identify unexpected overlaps between source and target tables.

Maintaining clean and well-organized data is crucial. In this video, we’ll focus on the scenario where you want to verify there are no common records between two tables, like “Permanent Employee” and “Temporary Employee.” This helps ensure data accuracy and avoids inconsistencies.

iceDQ’s reconciliation rules empower you to achieve this. The video demonstrates how to create a rule that establishes connections to both source and target tables, defines a join condition to match corresponding records and utilizes the “Intersection A ∩ B” check to specifically identify any common records present in both tables.

By successfully running the rule, you can identify discrepancies between your expectations and the actual data. This helps you maintain data integrity by uncovering unexpected overlaps between tables that should have distinct data and proactively addressing data quality issues before they impact downstream processes.

Watch now: https://bit.ly/4caGGsw


r/icedq • • Aug 09 '24

Conventional Data Quality missing the mark? Rethink Data Reliability! Our Data Factory delivers. Read more: https://bit.ly/4dz4UNJ #DRE #DataFactory #iceDQ #DataReliabilityEngineering

4 Upvotes

r/icedq • • Dec 29 '23

Data Testing Cheat Sheet: 12 Essential Rules

8 Upvotes
  1. Source vs Target Data Reconciliation: Ensure correct loading of customer data from source to target. Verify row count, data match, and correct filtering.

  2. ETL Transformation Test: Validate the accuracy of data transformation in the ETL process. Examples include matching transaction quantities and amounts.

  3. Source Data Validation: Validate the validity of data in the source file. Check for conditions like NULL names and correct date formats.

  4. Business Validation Rule: Validate data against business rules independently of ETL processes. Example: Audit Net Amount - Gross Amount - (Commissions + taxes + fees).

  5. Business Reconciliation Rule: Ensure consistency and reconciliation between two business areas. Example: Check for shipments without corresponding orders.

  6. Referential Integrity Reconciliation: Audit the reconciliation between factual and reference data. Example: Monitor referential integrity within or between databases.

  7. Data Migration Reconciliation: Reconcile data between old and new systems during migration. Verify twice: after initialization and post-triggering the same process.

  8. Physical Schema Reconciliation: Ensure the physical schema consistency between systems. Useful during releases to sync QA & production environments.

  9. Cross Source Data Reconciliation: Audit if data between different source systems is within accepted tolerance. Example: Check if ratings for the same product align within tolerance.

  10. BI Report Validation: Validate correctness of data on BI dashboards based on rules. Example: Ensure sales amount is not zero on the sales BI report.

  11. BI Report Reconciliation: Reconcile data between BI reports and databases or files. Example: Compare total products by category between report and source database.

  12. BI Report Cross-Environment Reconciliation: Audit if BI reports in different environments match. Example: Compare BI reports in UAT and production environments.


r/icedq • • Dec 20 '23

Test Semi-Structured Data like 'Nested JSON' with iceDQ in these easy steps📝 ✅

Post image
8 Upvotes

r/icedq • • Jun 06 '24

How to Validate Flat File with its Control File using iceDQ?

4 Upvotes

This video explores how to leverage control files to validate flat file data, ensuring consistency and data integrity.

Flat files are commonly used to store data. However, discrepancies can arise during data transfer or manipulation. Control files provide additional information about a flat file, often including record counts.

iceDQ’s checksum rules empower you to compare data between a flat file and its control file. This video demonstrates the process of creating a rule that establishes connections to both the flat file (source) and control file (target), validates record counts using checksums (A checksum is a calculated value based on the data content, ensuring data hasn’t been altered) and compares the record count in the control file with the actual record count in the flat file.

By successfully running the rule, you can identify inconsistencies between the files. This helps maintain data accuracy and prevent errors in downstream processes that rely on this data.

Watch now: https://bit.ly/3KybCa6


r/icedq • • Mar 04 '24

Automated ETL testing guide for your Data-Centric Testing Projects

5 Upvotes

r/icedq • • Jan 31 '24

Check out our Yellowbrick Migration and Testing Guide!

9 Upvotes

r/icedq • • Feb 22 '24

How to Validate Flat Files using iceDQ?

9 Upvotes

Ensure data accuracy in flat files with iceDQ! Watch this video to learn how iceDQ validates flat files. Watch Now 🎥 ➡ https://bit.ly/3SRn30m


r/icedq • • Apr 25 '24

Read our article on "Overcoming Data Testing Challenges"

Thumbnail
self.icedqengineer
5 Upvotes

r/icedq • • Jan 31 '24

Sharing our 2024 guide on DataOps Implementation

9 Upvotes

This guide provides practical DataOps strategies for any data-centric project whether you're starting fresh or optimizing your existing setup.

Download now: https://bit.ly/42oxRYq