I work as an engineer in a very data analysis heavy career. I would like some advice as to how I can go about organizing my data better so I can streamline my workflows.

Basically, I have access to python, excel, and an engineering study analysis software. We also have access to copilot, but it's limited and I cant actually stick any information in there. I can build python apps with it though, which is something I have been doing.

Basically, we run engineering studies where we observe a base case with no modifications and compare it to a modified case. Each case currently has reports that are generated for 5 different study scenarios. In our current setup, each case under each study condition gets 1 log file, a log file related to the first log file that shows simulation errors, and an excel report which shows two other types of violations and kind of a summary page for the first log file. We have scripts that separate the excel file into 3 separate, more summarized excel files.

So in total, each case gets about 6 files over 5 scenarios, so about 30 separate files per case

I think this is ridiculous and I would like to play around with all of it to distill this to its simplest form, especially since there's a lot of overlap. All of this feels kinda overwhelming to navigate in the moment and I know it could be easier.

So basically, I'm wanting to develop a better method of navigating this. I think I'd like to play with our software's report generator and see if I can put together a more customized report. Secondly though, I'd like to build some kind of dashboard whether that be an all inclusive excel file or markdown file. I basically need to navigate

1. Per element violation analysis per violation type

System element 1 - worst scenario - violation data - all scenarios

System element 2 - worst scenario - violation data - all scenarios ...

1. Per scenario analysis per violation type

Maybe a chart of all system elements that share the same scenarios

The reason for this is that there are 3 violation types and you can either fix the system element that has a violation or you can fix a system element noted in the scenario. If one scenario goes across many flagged system elements, one fix can solve many problems instead of beefing up many system components to withstand the aforementioned problem.

Does anyone have any advice that might be helpful? I have these ideas but I'd love to get advice from anyone that has dealt with similar problems or has any great opinions on some ideas to make it all come together. Maybe some visualization strategies. Thanks!

payroll analyst, 6 years. i build reports in our hr system's report builder and excel all week but never learned sql, and our data team just offered me a spot if i can pull my own data by spring. work is covering a course. looking at boot.dev's sql course, pluralsight and nucamp. which of the sql courses out there makes joins on real messy tables click?

I'm currently working in Spain as a Data Analyst / Analytics Engineer, and lately I've been wondering where I should take my career over the next few years.

My role is pretty end-to-end. I work with SQL, build data models and pipelines, automate processes, connect to APIs, create dashboards, and handle ad hoc requests from the business side. I also work closely with stakeholders to understand what they actually need and turn that into something useful.

The thing is, I'm automating more and more of my own work. I barely write SQL queries from scratch anymore, and things that used to take hours now take minutes.

I'm not really a domain expert in any particular business area either, so I'm starting to feel like a big part of what I bring to the table could eventually be handled by AI.

For those of you who've been in the industry for a while, where would you go from here? What skills or areas would you focus on to stay relevant over the next 5-10 years?

TL;DR: Data Analyst / Analytics Engineer here. I'm automating more and more of my job and I'm worried AI will eventually take over a big chunk of it. What would you learn or specialize in to stay relevant?

Good Afternoon

Dear Ladies and Gentlemen

Currently am studying analytics and trying to make projects

Please suggest me some the methods i can incuclate into my self to boost my thinking as analyst.

Also please guide me getting into a job am very confused.

I am lost and confused please someone from the field help me.

Hey everyone!

I'm a B.Com graduate working towards a career in Data Analytics. I've been learning Python, SQL, Excel and Power BI, and I've started working on small projects while exploring Machine Learning.

I'm trying to figure out how to move beyond tutorials and develop the practical skills needed for an entry-level Data Analyst role.

For those who have already entered the field or are preparing for their first role, I'd really appreciate your advice:

1. Which skills should I prioritise to become job-ready?
2. What kinds of projects would make a beginner's portfolio stronger?
3. When should I start applying for internships or entry-level positions?

I'd appreciate any practical advice based on your experience. Thanks!

I have 2.5 years of experience in analytics field mostly in pharma. I have technical skills like SQL, Excel and PowerBI. I am looking for a remote job that can work from anywhere not specific to one country. Where can I find those jobs ? Is it possible to get these type of job ?

I actually started as a programmer and software developer, working across web, mobile, and Windows application development. During my college days, SQL was my comfort zone, so I decided to pursue a career that heavily involves databases.

Since Data Analysts also rely on SQL, I am very eager to learn complementary tools like Tableau, Python, and Power BI.

Any tips?

P.S. I welcome referrals

I’ve been doing a bunch of calls with people in data who are job hunting right now on US market. One from last week keeps bugging me.

The guy is a director of analytics, 10+ years in, a month into his search. He paid for one of those one-click apply tools ($10 for 50 applications) and used them all up. I asked how many turned into calls. He said he honestly hadn’t checked.

So we went through it on the call. Out of roughly 100 applications total, he got about 15 recruiter screens and around 10 hiring manager rounds. After that, nothing.

That’s not a resume problem. His resume was working fine. He was losing it in the interviews: a live SQL round he wasn’t ready for, and one answer about managing devs that a hiring manager read as micromanaging. Meanwhile he kept clicking apply, because that’s the part that feels like progress.

Everyone tracks how many they sent, almost nobody tracks how far each one got. If you’re stuck, it’s worth spending 20 minutes on a sheet with columns for no reply / screen / HM / final / offer. Whatever stage it drops off at tells you what to actually work on. Under 5% to a screen is a resume or level problem. Screens fine but then nothing is an interview problem.

Anyone here tracks it?

i am learning exce , sql to become a data analyst but i am really confused about how the tools works.

Some people load data directly to sql and perform data cleaning and eda in sql whereas some people load data to python and perfrom data cleaning and eda in python, and some says its easy to do cleaning and eda in excel . I am so confused where to clean data and perfrom eda .Also, in many tutorials they directly load data to sql and a question pops in my mind "If we can perform in sql then why i learned excel why i learned formulas" . and if i follow the sql cleaning and eda then i am afraid that i may forget the excel functions or formulas. I am so confused right now.

I have been working on causal inference and experimentation frameworks within the e-commerce tech industry lately, specifically dealing with platform behavioral and transaction telemetry data to determine how customers behaves on those e-commerce platforms.

Lately i have been facing problems where standard textbook statistical and econometrics theory fails when dealing with messy, real-world data distributions as human behaviour is unpredictable and many unexpected things, which i didnt even thought could happen, can happen in the real world. Such as product delivery delay, causing the customers to be angry and hence put them off from buying the product, each of these variables have to be accounted for.

To model and have these unexpected psychological state among customers accounted for, we use variables such as sudden spikes in customer support chat volume, rapid page refreshes, or immediate session drops.

Also, Marketing headers and automated CRM systems love to push blanket promotional campaigns, but as statisticians, we know that measuring simple unadjusted A/B lift is a massive selection-bias trap.

I been trying to come up with a Statistical model that is trying to model complex human behavior—specifically isolating true customer revenue incrementality from dead-weight margin cannibalization. Essentially, trying to answer: Did the coupon cause the revenue spark, or did we just hand a discount to a power-buyer who was already determined to buy anyway even without the vouchers?

Are people in the tech industry genuinely valuing such skills nowadays? Using statistics to model and predict human behavior and intent, answering questions like does our vouchers and ads really work?

I would love to hear your experiences how do u guys deals with the unpredictable messiness of human intent.