NR-701 · Week 7 of 8 · Analyzing practice data over time

NR-701 Week 7 Analyzing Practice Data Over Time: How to Write It

The short answer

Late in an analytic methods course the object of appraisal usually shifts from published studies to the data a practice setting produces about itself. The doctoral skill is knowing what that data can and cannot support: how to define a measure so two people counting get the same answer, how to tell ordinary variation from a real signal, why comparing two averages before and after a change is the weakest thing you can do with a time series, and how to write about a quality improvement evaluation without dressing it up as a trial. Your section may print this as NR 701 or NR701; it is the same course. Chamberlain publishes no syllabi outside Canvas. The placement here is our teaching judgment from the course's catalog arc; your section's rubric decides what your week actually asks.

NR-701 Week 7 grading scale at Chamberlain, the criterion levels this assessment is scored on, from Chamberlain Tutors
How Chamberlain grades NR-701 Week 7, visualized by Chamberlain Tutors.

What NR-701 Week 7 asks for

A med-surg service compares its fall rate for the six months before a new hourly rounding standard with the six months after and reports an improvement. Plotted month by month across twenty-four months, the same data shows a rate that swings widely every month, with the pre-period happening to contain two unusually bad months and the post-period two unusually good ones. Nothing improved. The unit measured noise and named it a result, and it will spend a year defending a change that did nothing while the actual driver goes unexamined.

This is why time-series thinking matters more than statistical testing in practice work. Data from a service is not a sample drawn from a population; it is a process observed repeatedly, and processes have inherent variation. Common cause variation is the ordinary fluctuation a stable process produces, and reacting to it is the most common analytic error in health care management. Special cause variation is a signal that something changed, and it is detectable by rules about runs, trends, shifts and points beyond expected limits rather than by comparing two means.

The measure family matters as much as the analysis. Outcome measures report what happened to patients. Process measures report whether the change was actually done, which is what distinguishes a failed intervention from an intervention that never occurred. Balancing measures report what got worse elsewhere, since almost every change moves work somewhere. A practice evaluation that reports only outcomes cannot explain its own result.

Data quality deserves a section of its own. Every measure depends on a field that a human being populates under time pressure, and the operational definition is only as good as the documentation behind it. Before analyzing anything, a doctoral student should be able to say who enters the data, when, from what prompt, and what happens when the situation does not fit the options offered. Expect a written analysis or evaluation plan for practice data, often with a chart, and possibly a board post about a measure definition. Keep any real data aggregated and de-identified, and follow your organization's rules on what may be shared outside it; posts do not reopen once submitted in Canvas.

The NR-701 Week 7 method, step by step

Six analytic moves for reading a service's own numbers honestly.

  1. Write the operational definition before you request any data

    Numerator, denominator, inclusions, exclusions, the time window and the data source, specified so two analysts would count identically. Most disputes about whether a change worked are really disputes about a definition nobody wrote down.

  2. Assemble outcome, process and balancing measures as a set

    Name at least one of each and say what each would show. A process measure is what lets you tell a change that failed from a change that was never delivered, and its absence is the most common gap in student evaluation plans.

  3. Plot the data over time before you calculate anything

    Use enough points to see the process, which usually means twenty or more intervals. A plot answers questions about trend, seasonality and instability that no summary statistic will show you.

  4. Apply stated rules for signal detection

    Runs above or below the centre line, sustained trends, and points outside expected limits are the recognized signals. Say which rules you are using and cite the source, so a reader can distinguish a detected signal from an eyeballed one.

  5. Interrogate the denominator and the case mix

    A rate per 1,000 patient days behaves differently from a rate per admission, and a unit whose acuity rose during the period has a different baseline than the one it started with. Say what the denominator is and what could have shifted underneath it.

  6. Write the limits of what the evaluation can claim

    State plainly that this is an evaluation of a change in one setting, that concurrent changes cannot be excluded, and what a reader should therefore not conclude. That paragraph is a mark of doctoral judgment, not a weakness.

A layout and word budget for a practice data analysis

Our frame for analyzing or planning to analyze service data, sized for roughly 1,600 to 2,000 words plus any chart. It is our own outline rather than anything the university issues, and your week's rubric outranks it wherever the two disagree.

SectionWhat belongs in itWord target
The question the data must answerWhat decision depends on this analysis and what result would change it.160 to 200
Measure setOutcome, process and balancing measures named, each with its operational definition written out.340 to 420
Data source and qualityWhere each measure comes from, who enters it, when, and the failure modes of that documentation.280 to 340
Display over timeThe plotting approach, the interval chosen, the number of points, and what the baseline period shows.260 to 320
Signal rules and findingsThe detection rules applied with their source, and what the data does or does not show against them.300 to 360
Denominator and case mixWhat the rate is per, and what could have shifted underneath it during the period.200 to 250
What can and cannot be concludedThe claim the evaluation supports, the rival explanations, and the language a report should avoid.220 to 280

Evidence craft for practice data

Keep quality improvement language distinct from research language. An evaluation of a local change is not a trial, has no arms and enrolls no subjects. Using research vocabulary for improvement work is both inaccurate and, in a course about analytic methods, immediately visible.

Cite a published reporting standard for improvement work. Standards exist for describing improvement projects, and naming one shows a grader that your write-up follows a recognized structure rather than an improvised one.

Report counts and denominators for every interval. Rates built on small denominators swing violently, and a month with three events out of 190 patient days should be shown as such rather than converted to a rate presented with false precision.

Protect the data and the people in it. Aggregate, de-identify, avoid cell sizes small enough to identify an individual patient or staff member, and follow your organization's rules about what may leave it. Where you cannot share actual figures, describe the measure structure instead.

Make no claim about oversight determinations. Whether a project counts as quality improvement or requires review is decided by the institution and its review board. Say the determination will be sought and describe the protections you would apply regardless of the answer.

Five mistakes that cost points in this week's territory

  • Two averages compared. Before and after means discard the shape of the data and manufacture results out of ordinary variation.
  • No process measure. Without one, a null result cannot be explained and the evaluation cannot tell whether the change happened at all.
  • Undefined measures. A numerator and denominator that two people would count differently makes every later number unusable.
  • Ignoring the denominator's behavior. A falling rate produced by a rising denominator is a case-mix story, not an improvement story.
  • Trial vocabulary for improvement work. Subjects, arms, randomization and hypothesis testing misdescribe what a practice evaluation is and how much it can claim.

Before you submit

  • Every measure has a written operational definition with numerator and denominator
  • Outcome, process and balancing measures all appear
  • The data source and who populates it are described
  • Data is displayed over time with enough points to show the process
  • Signal detection rules are named and attributed
  • Rival explanations for any observed change are stated
  • All figures are aggregated and de-identified, and no oversight outcome is predicted

Analyzing practice data for NR-701?

Send the rubric and your measure set out of Canvas. A premium original draft comes back in 24 to 48 hours with definitions written out and the variation read correctly, and revisions run until the grade lands.

Questions students ask about this stage

I only have twelve months of data. Is that enough to plot?
It is enough to start, and you should say what the limitation costs you. Twelve monthly points give a thin baseline, which means a shift has to be large before you can distinguish it from ordinary variation, and seasonal effects are impossible to separate from real change. Two adjustments help. Consider a shorter interval where the volume allows it, since weekly or biweekly points give you more observations from the same period without inventing data. And be explicit that your baseline is short, so any conclusion is provisional and should be revisited once more points accumulate. A doctoral reader accepts a short series described honestly far more readily than a confident conclusion drawn from it.
My unit's numbers are so small that the rate jumps every month. What do I do?
Small denominators are the normal condition on a single unit and they call for different displays rather than for giving up. Consider plotting the count of events instead of a rate when the denominator is stable, or the number of cases between events when the event is rare, since that measure becomes more sensitive as performance improves rather than less. Widening the interval to quarters trades sensitivity for stability and is sometimes the honest choice. Whatever you choose, resist the temptation to report a rate with decimal places that the underlying counts cannot support, and say in the limitations that the unit's volume constrains what any analysis can detect.
Does an improvement evaluation need statistical testing at all?
Often not, and reaching for a significance test is frequently a sign that the data has not been plotted. Improvement work is usually best served by displaying the process over time and applying stated signal rules, because that approach answers the question a service actually has: did something change, and when. Statistical tests answer a different question about sampling and can be misleading when applied to serially correlated observations from one process. Where your rubric or your site asks for a test, use one that suits time-ordered data and say what assumption it makes. Either way, describe what you did precisely, because in this course the reasoning behind the analytic choice is worth more than the choice itself.

Keep going

Online now