Opened 6 years ago

Closed 5 years ago

Last modified 4 years ago

#833 closed enhancement (fixed)

multicohort survival analysis plugin

Reported by: rwaitman Owned by: dconnolly
Priority: major Milestone: heron-elk-city-update
Component: data-repository Keywords: public-web
Cc: badagarla, ngraham, dconnolly Blocked By: 1104, 1226
Blocking: 1625 Sensitive: no

Description

This will expand upon survival for the cancer population to include any start event and any outcome (not just death)

compare two or more cohorts and plot them.

Change History (30)

comment:1 Changed 6 years ago by dconnolly

I'm using HeronStatsPlugins to explore the surrounding design space;
e.g. I summarized the whiteboard discussion yesterday about various ways to connect HERON to R.

comment:2 Changed 6 years ago by dconnolly

  • Milestone changed from heron-clinton-update to heron-eldorado-update
  • Type changed from task to enhancement

comment:3 Changed 6 years ago by dconnolly

  • Milestone changed from heron-eldorado-update to nci-survival

comment:4 Changed 6 years ago by dconnolly

  • Status changed from new to assigned

made a little progress:

It's deployed on the development server. It doesn't really do anything.

comment:5 Changed 6 years ago by dconnolly

  • Milestone changed from heron-eldorado-update to nci-survival

comment:6 Changed 6 years ago by dconnolly

  • Milestone changed from nci-survival to HERONv.Next

comment:7 Changed 5 years ago by dconnolly

  • Milestone changed from HERONv.Next to Copper-John
  • Owner changed from dconnolly to srao2

Here are some starting examples discussed with Russ and Shanker:

  • A classic cancer copper john scenario would be comparing survival for two breast cancer populations (say obese versus non-obese)
    • for Attractor Nymph, you'd now be able to compare prostate cancer survival versus lung cancer survival
  • Non cancer:
    • survival (death) from hip surgery (from UHC) entrance criteria is admission to hospital for a specific MS-DRG and
      • outcome is death, or
      • readmission.
    • entrance criteria is is date of birth,
      • outcome is development of a diagnosis such as a heart attack AMI (generalize the Italian work) or
      • diabetes

comment:8 Changed 5 years ago by dconnolly

  • Blocked By set to 1225

comment:9 Changed 5 years ago by dconnolly

  • Blocked By changed from 1225 to 1225, 1226

(In #1226) Actually, this is more of a step along the way to #833 than a dup.

comment:10 Changed 5 years ago by dconnolly

  • Blocked By changed from 1225, 1226 to 1226, 1302

comment:12 Changed 5 years ago by dconnolly

  • Milestone changed from Copper-John to heron-cedarbluff-update

We'll synchronize deployment to production with cedarbluff.

comment:13 Changed 5 years ago by dconnolly

I spent a chunk on this on Monday; e.g. [921eaddc7b5d/rgate].

comment:14 Changed 5 years ago by dconnolly

  • Resolution set to fixed
  • Status changed from assigned to closed

Testing (as part of deployment #1302) showed that this is working.

comment:15 Changed 5 years ago by dconnolly

  • Resolution fixed deleted
  • Status changed from closed to reopened

umm... this didn't fit into the release just yet...

comment:16 Changed 5 years ago by dconnolly

  • Milestone changed from heron-cedarbluff-update to heron-waconda-update

I thought I could just cherry-pick the changeset for the new plot output directory, but it seems to have gotten mixed up with other stuff [11af89bae1a7].

comment:17 Changed 5 years ago by dconnolly

  • Owner changed from srao2 to dconnolly
  • Priority changed from major to minor
  • Status changed from reopened to assigned

Russ, in our meeting last Friday morning about HeronStatsPlugins, I was surprised by the version of Copper-John that's in production. I shouldn't have been. A week earlier, when Nathan and I tried to wrap up Cedar Bluff, we didn't manage to wrap up this part, so we re-opened it. (comment:15).

Maybe I can sort through it for Waconda, but I don't think it's critical and my plate is pretty full, so I'm marking it minor priority.

comment:18 Changed 5 years ago by dconnolly

  • Milestone changed from heron-waconda-update to heron-walnut-update

Updating the production deployment doesn't really help as long as pretty much all queries time out. (#845)

Also, I think it's important to have automated tests in place. (#1104)

comment:20 Changed 5 years ago by dconnolly

  • Blocking set to 1466
  • Cc ngraham added
  • Priority changed from minor to major

Nathan, Bhargav, now that I have a basic automated test (#1104), what's left is production ops/deployment stuff, for which I'd like the usual extra set of eyeballs. As we get closer to release (#1466), I'll look for a time.

comment:21 Changed 5 years ago by dconnolly

  • Status changed from assigned to accepted

I pruned a bunch of dead code: about 400 of the 600 lines [1ed27d84b43e/kmstat]. And there's still a lot of redundancy to be factored out.

comment:22 Changed 5 years ago by dconnolly

[9d38643eba74/kmstat] is now in production.

I'm not very confident about the output, but I'm going to claim victory, for now.

comment:23 Changed 5 years ago by dconnolly

  • Resolution set to fixed
  • Status changed from accepted to closed

comment:24 Changed 5 years ago by dconnolly

  • Milestone changed from heron-walnut-update to heron-elk-city-update
  • Resolution fixed deleted
  • Status changed from closed to reopened

In fixing performance issues (#845) I made some API changes [0349d1687f55/rgate] and I have yet to update source:rgate/rgate/cj_analysis.R to match.

comment:25 Changed 5 years ago by dconnolly

  • Status changed from reopened to accepted

comment:26 Changed 5 years ago by dconnolly

  • Resolution set to fixed
  • Status changed from accepted to closed

After refactoring the test harness to take advantage of the Data Builder for R (#1485) and reviewing the whole design and known issues using RStudio, it's working again as of [3be31a373593/rgate] and [51e64a655dc2/kmstat].

A couple related tickets remain:

  • make 2nd cohort optional #1676
  • finish up new chi-squared statistic UI #1347

comment:27 Changed 5 years ago by dconnolly

  • Blocking changed from 1466 to 1625

Let's deploy the new rgate and kmstat for Elk City.

comment:28 Changed 5 years ago by dconnolly

  • Blocked By changed from 1226, 1302 to 1104, 1226

pruning historical dependencies a bit for Elk City planning

comment:29 Changed 4 years ago by dconnolly

  • Keywords public-web added

comment:30 Changed 4 years ago by ngraham

  • Cc dconnolly added

comment:31 Changed 4 years ago by ngraham

Replying to dconnolly:

(In #1104) It worked, i.e. tested both the KM and CJ plugins, in Nathan's environment, though it's a bit flakey.

There's still a glitch in grabbing the patient set for a query in the history. Sometimes a click gets lost and it hangs. But if you click on the right query yourself (try several) it will resume on its own.

Getting it running was something like:

  1. cd kmstat/plugin_tests
  2. pip install -r requirements.txt
  3. python surv_multi1.py https://<server>/heron/

Dan's comment was in comment:19. However, it got corrupted when I tried to edit the ticket in preparation for publication. So, copying it here.

comment:32 Changed 4 years ago by kcrane2

Approved for public release.

Note: See TracTickets for help on using tickets.