Percentage-Change Approach to Clinical Significance in R

Introduction

In this tutorial, we will explore the percentage-change approach to clinical significance using R. The percentage-change approach is centered around a predefined relative change, expressed in percent, that needs to be achieved in order to inferr a clinically significant change. If, for instance, the percentage-change cutoff is believed to be 30%, and a patient demonstrated a score change of at least 30%, then this change is believed to be clinically significant. We will be working with the antidepressants and claus_2020 datasets, and the cs_percentage() function to demonstrate various aspects of this approach.

Prerequisites

Before we begin, ensure that you have the following prerequisites in place:

Looking at the Datasets

First, let’s have a look at the datasets, which come with the package.

library(clinicalsignificance)

antidepressants
#> # A tibble: 1,110 × 4
#>    patient condition measurement mom_di
#>    <chr>   <fct>     <fct>        <dbl>
#>  1 S001    Wait List Before          50
#>  2 S001    Wait List After           36
#>  3 S002    Wait List Before          40
#>  4 S002    Wait List After           32
#>  5 S003    Wait List Before          38
#>  6 S003    Wait List After           41
#>  7 S004    Wait List Before          29
#>  8 S004    Wait List After           44
#>  9 S005    Wait List Before          37
#> 10 S005    Wait List After           45
#> # ℹ 1,100 more rows
claus_2020
#> # A tibble: 172 × 9
#>       id   age sex    treatment  time   bdi shaps   who  hamd
#>    <dbl> <dbl> <fct>  <fct>     <dbl> <dbl> <dbl> <dbl> <dbl>
#>  1     1    54 Male   TAU           1    33     9     0    25
#>  2     1    54 Male   TAU           2    28     6     3    17
#>  3     1    54 Male   TAU           3    28     9     7    13
#>  4     1    54 Male   TAU           4    27     8     3    13
#>  5     2    52 Female PA            1    26    11     2    15
#>  6     2    52 Female PA            2    26    10     0    16
#>  7     2    52 Female PA            3    25    10     0     7
#>  8     2    52 Female PA            4    19     9     3    11
#>  9     3    54 Male   PA            1    15     2     0    28
#> 10     3    54 Male   PA            2    13     5     9    17
#> # ℹ 162 more rows

Percentage-Change Approach

The cs_percentage() function is a tool for assessing clinical significance. It allows you to determine if changes in patient outcomes are practically significant. Let’s go through the basic usage and some advanced features of this function.

Basic Analysis

Let’s start with a basic clinical significance distribution analysis using the antidepressants dataset. We are interested in the Mind over Mood Depression Inventory (mom_di) measurements and want to set the percentage cutoff for an improvement to a value of 0.3.

pct_results <- antidepressants |> 
  cs_percentage(
    id = patient,
    time = measurement,
    outcome = mom_di,
    pct_improvement = 0.3
  )
#> ℹ Your "Before" was set as pre measurement and and your "After" as post.
#> • If that is not correct, please specify the pre measurement with the argument
#>   "pre".

Handling Warnings

Sometimes, as in the example above, you may encounter warnings when using this function. You can turn off the warning by explicitly specifying the pre-measurement time point using the pre parameter. This can be helpful when your data lacks clear pre-post measurement labels.

# Turning off the warning by specifying pre-measurement time
pct_results <- antidepressants |>
  cs_percentage(
    id = patient,
    time = measurement,
    outcome = mom_di,
    pre = "Before",
    pct_improvement = 0.5
  )

Here’s a breakdown of the code:

Printing and Summarizing the Results

# Print the results
pct_results
#> 
#> ── Clinical Significance Results ──
#> 
#> Percentage-change approach with a 50% decrease in instrument scores indicating
#> a clinical significant improvement.
#> Category     |   n | Percent
#> ----------------------------
#> Improved     | 155 |  27.93%
#> Unchanged    | 366 |  65.95%
#> Deteriorated |  34 |   6.13%

# Get a summary
summary(pct_results)
#> 
#> ── Clinical Significance Results ──
#> 
#> Percentage-change analysis of clinical significance with a 50% decrease in
#> instrument scores indicating a clinical significant improvement.
#> There were 555 participants in the whole dataset of which 555 (100%) could be
#> included in the analysis.
#> 
#> ── Individual Level Results
#> Category     |   n | Percent
#> ----------------------------
#> Improved     | 155 |  27.93%
#> Unchanged    | 366 |  65.95%
#> Deteriorated |  34 |   6.13%

Visualizing the Results

Visualizing the results can help you better understand the clinical significance of changes in patient outcomes.

# Plot the results
plot(pct_results)


# Show clinical significance categories
plot(pct_results, show = category)

Data with More Than Two Measurements

When working with data that has more than two measurements, you must explicitly define the pre and post measurement time points using the pre and post parameters.

# Clinical significance distribution analysis with more than two measurements
cs_results <- claus_2020 |>
  cs_percentage(
    id,
    time,
    bdi,
    pre = 1,
    post = 4,
    pct_improvement = 0.3
  )

# Display the results
cs_results
#> 
#> ── Clinical Significance Results ──
#> 
#> Percentage-change approach with a 30% decrease in instrument scores indicating
#> a clinical significant improvement.
#> Category     |  n | Percent
#> ---------------------------
#> Improved     | 19 |  47.50%
#> Unchanged    | 21 |  52.50%
#> Deteriorated |  0 |   0.00%
summary(cs_results)
#> 
#> ── Clinical Significance Results ──
#> 
#> Percentage-change analysis of clinical significance with a 30% decrease in
#> instrument scores indicating a clinical significant improvement.
#> There were 43 participants in the whole dataset of which 40 (93%) could be
#> included in the analysis.
#> 
#> ── Individual Level Results
#> Category     |  n | Percent
#> ---------------------------
#> Improved     | 19 |  47.50%
#> Unchanged    | 21 |  52.50%
#> Deteriorated |  0 |   0.00%
plot(cs_results)

Setting Different Thresholds

You can set different thresholds for improvement and deterioration by adjusting the pct_deterioration argument Let’s see an example:

# Clinical significance analysis with different improvement and deterioration thresholds
cs_results_2 <- claus_2020 |>
  cs_percentage(
    id = id,
    time = time,
    outcome = hamd,
    pre = 1,
    post = 4,
    pct_improvement = 0.3,
    pct_deterioration = 0.2
  )

# Display the results
cs_results_2
#> 
#> ── Clinical Significance Results ──
#> 
#> Percentage-change approach with a 30% decrease in instrument scores indicating
#> a clinical significant improvement and a 20% increase in instrument scores
#> indicating a clinical significant deterioration.
#> Category     |  n | Percent
#> ---------------------------
#> Improved     | 25 |  62.50%
#> Unchanged    | 14 |  35.00%
#> Deteriorated |  1 |   2.50%

# Visualize the results
plot(cs_results_2)

Grouped Analysis

You can also perform a grouped analysis by providing a group column from the data. This is useful when comparing treatment groups or other categories.

cs_results_grouped <- claus_2020 |>
  cs_percentage(
    id = id,
    time = time,
    outcome = hamd,
    pre = 1,
    post = 4,
    pct_improvement = 0.3,
    group = treatment
  )

# Display and visualize the results
cs_results_grouped
#> 
#> ── Clinical Significance Results ──
#> 
#> Percentage-change approach with a 30% decrease in instrument scores indicating
#> a clinical significant improvement.
#> Group |     Category |  n | Percent
#> -----------------------------------
#> TAU   |     Improved |  6 |  15.00%
#> TAU   |    Unchanged | 13 |  32.50%
#> TAU   | Deteriorated |  0 |   0.00%
#> PA    |     Improved | 19 |  47.50%
#> PA    |    Unchanged |  2 |   5.00%
#> PA    | Deteriorated |  0 |   0.00%
plot(cs_results_grouped)

Analyzing Positive Outcomes

In some cases, higher values of an outcome may be considered better. You can specify this using the better_is argument. Let’s see an example with the WHO-5 score where higher values are considered better.

# Clinical significance analysis for outcomes where higher values are better
cs_results_who <- claus_2020 |>
  cs_percentage(
    id = id,
    time = time,
    outcome = who,
    pre = 1,
    post = 4,
    pct_improvement = 0.3,
    better_is = "higher"
  )

# Display the results
cs_results_who
#> 
#> ── Clinical Significance Results ──
#> 
#> Percentage-change approach with a 30% increase in instrument scores indicating
#> a clinical significant improvement.
#> Category     |  n | Percent
#> ---------------------------
#> Improved     | 28 |  70.00%
#> Unchanged    |  9 |  22.50%
#> Deteriorated |  3 |   7.50%

Conclusion

In this tutorial, you’ve learned how to perform clinical significance analysis using the cs_percentage() function in R. This analysis may be crucial for determining the practical importance of changes in patient outcomes. By adjusting thresholds and considering grouped analyses, you can gain valuable insights for healthcare and clinical research applications.