Many organizations build dashboards around KPIs that were defined years ago.

Over time, teams align targets, incentives, and reporting processes around those metrics. The KPI becomes part of how performance is measured.

The challenge is that businesses evolve.

Customer behavior changes.
Products change.
Markets change.
Business priorities change.

Yet many KPIs remain untouched.

As a Business Analyst, how do you determine when a KPI is no longer driving the right decisions?

What signals tell you that a metric has become a reporting exercise rather than a meaningful measure of business performance?

Have you ever had to retire a KPI that leadership was heavily invested in, and how did you justify the change?

As embedding models continue to improve, many teams are solving domain-specific NLP problems using retrieval, semantic search, and RAG pipelines instead of fine-tuning foundation models.

This raises an interesting technical question:

At what point does improving retrieval stop being enough?

For tasks involving domain expertise, specialized terminology, reasoning, or long-context understanding, when would you choose:

• Better embeddings
• Better retrieval
• Fine-tuning
• A combination of both

Have you encountered production use cases where retrieval-based approaches hit a ceiling that only fine-tuning could overcome?

Or do you think increasingly powerful embedding models are making fine-tuning less necessary for most enterprise NLP workloads?

I recently saw a discussion where a model achieved surprisingly high validation performance, only to fail badly in production.

The root cause wasn’t the algorithm, it was feature engineering.

Some of the engineered features were technically available during training but relied on information that wouldn’t exist at prediction time.

The line between clever feature engineering and subtle leakage isn’t always obvious.

How do you evaluate whether a feature is introducing future information, target leakage, or unrealistic signals?

Have you encountered cases where leakage survived code reviews and validation checks?

I recently saw a discussion where a model achieved surprisingly high validation performance, only to fail badly in production.

The root cause wasn’t the algorithm, it was feature engineering.

Some of the engineered features were technically available during training but relied on information that wouldn’t exist at prediction time.

The line between clever feature engineering and subtle leakage isn’t always obvious.

How do you evaluate whether a feature is introducing future information, target leakage, or unrealistic signals?

Have you encountered cases where leakage survived code reviews and validation checks?

I’m not asking whether AI can replace every data scientist tomorrow.

What I’m wondering is whether we’re underestimating how much the role could change over the next 5–10 years.

Tasks that once required technical expertise, writing SQL, generating code, creating dashboards, building basic models—are becoming increasingly accessible through AI tools.

For someone entering the field today, it’s hard not to wonder:

• Which skills will still be valuable?
• Will companies need fewer data professionals?
• Will data science become more of a business role than a technical one?
• How do you stay relevant when AI keeps lowering the barrier to entry?

Curious to hear from people working in analytics, data science, engineering, and AI.

What part of the profession do you think becomes more valuable in an AI-first world—and what part becomes less valuable?