Opened 5 years ago

Closed 5 years ago

Last modified 4 years ago

#1304 closed enhancement (fixed)

drag-and-drop entrance criteria, outcome, censor for survival analysis plug-in

Reported by: dconnolly Owned by: dconnolly
Priority: major Milestone: heron-elk-city-update
Component: data-repository Keywords: public-web
Cc: badagarla Blocked By:
Blocking: Sensitive: no

Description

Attachments (5)

SeparateConcept.png (91.8 KB) - added by srao2 5 years ago.
generalized.png (6.8 KB) - added by srao2 5 years ago.
survdiff.png (7.7 KB) - added by srao2 5 years ago.
StratificationAutoLabel.png (38.1 KB) - added by srao2 5 years ago.
KM_ethnicity.png (7.8 KB) - added by srao2 5 years ago.

Download all attachments as: .zip

Change History (38)

comment:1 Changed 5 years ago by srao2

Plugin Attractor Nymph is created on the development server - this is currently just a clone of Copper John. I will be adding enhancements onto it as per our discussion yesterday.

comment:2 Changed 5 years ago by dconnolly

I was thinking about whether to make a separate plugin... I don't think we want to maintain both Copper John and a new one. I suggest you just keep working on Copper John.

comment:3 follow-up: Changed 5 years ago by srao2

Alright, that is fine with me. I will add newer enhancements to Copper John only.
Dan, do you have a reference or link to the Italian study we discussed earlier?

Changed 5 years ago by srao2

comment:4 Changed 5 years ago by srao2

Within Copper-John plugin, I have implemented drag-and-drop facility for concept site information for each patient-set separately [2548e004302d/kmstat]. attachment:SeparateConcept.png is a representative screenshot from the development environment.

comment:5 in reply to: ↑ 3 Changed 5 years ago by dconnolly

Replying to srao2:

Dan, do you have a reference or link to the Italian study we discussed earlier?

Yes, it's the paper by Segagni et. al. that it cited from the plugin page itself, as well as from KUBMIPresentations#useR2012-06 and from the R Integration announcement.

comment:6 Changed 5 years ago by srao2

comment:7 Changed 5 years ago by srao2

Cosmetic changes in the display page that reflect current status of functionality available in the Copper-John plugin were made [f1f2b45e94a8/kmstat]. I will continue further steps towards generalization of other criteria in follow-up ticket(s).

comment:8 Changed 5 years ago by srao2

Continuing work towards generalization of outcome and censoring event. As for first step, I have implemented these two options with drag-and-drop facility. This task has not finished and I haven't tested it with data yet.

comment:9 Changed 5 years ago by srao2

For the implementation of drag-and-drop facility with 'outcome' and 'censoring event', first I commented-out radio button implementation and then added drag-and-drop code in injected_screens.html. After that some code was added in the CopperJohnStat_ctrlr.js to pick the dropped data and post to Rgate. This seems to be working fine on the development server.
Re-implementation of radio button option alongside drag-drop has been with limited success so far. The strategy I am working on is to use javascript onlick option where by default radio button option would be visible whereas onclick drag-and-drop field would appear with simultaneous disappearance of radio button. As of now, disappearing of radio button is not working.

comment:10 Changed 5 years ago by srao2

As per Dan's advice, I used jQuery to implement drag-drop and radio-button options side-by-side for outcome and censoring event. Preliminary observation indicates that the implementation is working alright. I am not sure if some events are ineffective due to development environment or if there is some bug. I still need to test implementation with test data and tweak few things... will update here when it works better.

comment:11 Changed 5 years ago by srao2

For the time being, radio button options have been disabled (per Dr. Waitman's advice). After finishing up with 'Outcome' and 'Censor' drag-drop implementation, I have repeated steps for 'Stratification' as well. All modifications have been updated to the kmstat and rgate repositories of version control system.

[95536489f1a2/kmstat], [f5a49e90a629/rgate]

comment:12 Changed 5 years ago by srao2

As a next step, I am trying to implement a mechanism for comparison of survival curves - something similar as explained in this link: http://www.gigawiz.com/kaplan.html
I was hoping to integrate into our survival analysis plugin, a chi-square based statistic (e.g., logrank) alongwith significance level for convenient comparison.

comment:13 Changed 5 years ago by srao2

For the comparison of two survival curves, I have implemented significance test using function survdiff() of survival package. Details: For each two patient dataset, I introduced another column that uniquely identifies its rows and then I combined two datasets together. The newly introduced column differentiates rows from two patient sets after this combine operation. After analysis, survdiff computes chi square statistic and p-value.

As per chi square distribution table (http://home.comcast.net/~sharov/PopEcol/tables/chisq.html), for 1 degrees of freedom and 0.05 alpha significance level, critical chi square value is 3.84. In the worked example below, chi square value is greater than 3.84 and p-value is less than 0.05 - which imply that we reject null hypothesis and conclude that there is a strong evidence in favor of alternal hypothesis. In other words, two curves are significantly different from each other.

Call:
survdiff(formula = Surv(input$time, input$outcome) ~ pSet, data = input)

N Observed Expected (O-E)2/E (O-E)2/V

pSet=A 43 12 19.7 2.98 6.38
pSet=B 50 27 19.3 3.03 6.38

Chisq= 6.4 on 1 degrees of freedom, p= 0.0115

comment:14 Changed 5 years ago by srao2

For some reasons, I am not able to test my implementation on the development server. Last time when I checked, front-end implementation was working fine and generated plot successfully. Now I am getting this error message:

error from R gateway:
exception from plot_survival:Error in terms.formula(formula, data = data) : 
  invalid model formula in ExtractVars

Comparison statistic implemented via R-script is working fine on my desktop computer and I have update the same on the development server and version control system.

comment:15 Changed 5 years ago by dconnolly

I was able to reproduce the problem from the command-line integration test harness, using the current version [c64e66cd1934/rgate] with cj_analysis.R modified as in ~srao2/projects/rgate/rgate.

The version checked out in ~srao2/projects/rgate/ is about 3 weeks old; its test harness doesn't work.

The difference between Monday and now is perhaps that the temporary kludge for test data (ticket:1271#comment:9) got wiped out while testing fixes to HERON this week. The script could perhaps give better diagnostics in this case.

Hmm... the problem persists even after we restored the test data. My only suggestion is to step through the R code line by line and see what's happening.

comment:16 Changed 5 years ago by srao2

All works well now :)
I will make some cosmetic changes R script and update on version control. Attached image is the one just produced.

Changed 5 years ago by srao2

comment:17 Changed 5 years ago by srao2

For the time being, I have put chi-square value, d.f. and p-value on a separate legend box at bottomleft on the plot itself. Representative figure is attached: attachment:survdiff.png. I will incorporate these changes on the development server before production deployment.

Changed 5 years ago by srao2

comment:18 Changed 5 years ago by dconnolly

  • Resolution set to fixed
  • Status changed from new to closed

The current code ([a4b32e4ed586/rgate], [6d233e8ebadb/kmstat]) meets requirements discussed in HeronStatsPlugins#GeneralizedSurvivalAnalysis, short of radio buttons for common cases; these are less critical, according to recent discussion with Russ. No attempt was made to address any of the "bonus points" or challenging issues.

Misc. review notes, none of which seems critical:

  • usability
    • radio buttons for common outcomes?
    • radio buttons for common stratification?
    • make stratification optional?
  • layout
    • move help text to the right of the controls? to a separate tab?
    • use 2 columns for patient set 1/2?
  • coding style
    • re-factor repetitive code

comment:19 Changed 5 years ago by dconnolly

  • Resolution fixed deleted
  • Status changed from closed to reopened

On Tue, 2012-08-28 at 15:14 +0000, Russ Waitman wrote:

...
Minor point: widen some buttons where the text is being cut off
...
It shouldn't require stratification.

  • but, we may want to color code and label the stratification choices.

...
I tried to run it without a censoring event and hit seems to hang.

Changed 5 years ago by srao2

comment:20 Changed 5 years ago by srao2

Drop-box width is increased by editing line 45 in file - vwCopperJohnStat.css
Labels for stratified sets are automated so that color and type of curve is faithfully labeled in the legend box. Here is a representative plot - attachment:StratificationAutoLabel.png

comment:21 Changed 5 years ago by srao2

So far I am not able to simultaneously implement radio button and drag-drop box. I will try to resolve this over the weekend.

Changed 5 years ago by srao2

comment:22 Changed 5 years ago by srao2

Simultaneous implementation of radio-button and drag-drop box has been accomplished. Now, we can either choose one of the radio-button options or drag some other criterion of our interest into the drop-box. Also included is the 'none' option for stratification and censor. None works fine for stratification but in case of censor, it leads to error message stating 0 patients selected. A representative plot is attached (lymphoma and leukemia stratified on ethnicity)- attachment:KM_ethnicity.png
I believe, the tool will behave better if we insert little larger dataset. I am thinking of how to handle censoring event...

comment:23 follow-up: Changed 5 years ago by srao2

We deployed new implementation to another development server but unfortunately plug-in crashed before generating plot. I noticed that previous development deployment was also not working for me. Following error message is obtained:

error from R gateway:

...
    Error message:
    <br />Premature end of script headers: rgate.cgi
...
  <span>Mon Sep 10 15:52:56 2012<br />
  Apache</span>

comment:24 Changed 5 years ago by dconnolly

  • Cc badagarla added

That's a very generic error message; try to get more details from the apache logs:

This message says "your script broke and it didn't send any HTTP headers before sending the error messages". Some suggestions for further investigation:

  • Check the error message that was written in your apache error log.
  • ...

-- How to get over 'premature end of script headers' in Apache? July 2000

p.s. also noted as ApacheTroubleShooting#cgi-script-error?

comment:25 in reply to: ↑ 23 Changed 5 years ago by dconnolly

Replying to srao2:

    <br />Premature end of script headers: rgate.cgi
...
  <span>Mon Sep 10 15:52:56 2012<br />
  Apache</span>

Try it now?

I finally got around to fixing the development rgate installation (with help from Nathan... ticket:822#comment:4).

comment:26 Changed 5 years ago by srao2

Thanks, it does work now. I tried with data consisting for approx 300-400 patient sets and it did produce plot. But when I tried with larger data e.g., > 1000 patient sets, it gives following error message:

error from R gateway:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>504 Gateway Time-out</title>
</head><body>
<h1>Gateway Time-out</h1>
<p>The gateway did not receive a timely response
from the upstream server or application.</p>
</body></html>

comment:27 Changed 5 years ago by dconnolly

Yes, that's #845.

I wonder if we're ready to close this one... I don't have time to think it through just now.

comment:28 Changed 5 years ago by srao2

Yes, I think this one should be closed. There seems to little more to do in #1350 as KM_plot as obtained on my desktop computer is not similar to that from development server deployment.

comment:29 Changed 5 years ago by dconnolly

Well, we made some progress ([71a8aa8c10ff/rgate]), but the output still doesn't look right.

comment:30 Changed 5 years ago by dconnolly

  • Milestone changed from Attractor-Nymph to heron-elk-city-update
  • Owner changed from srao2 to dconnolly
  • Status changed from reopened to accepted

I don't think drag-and-drop stratification makes sense in this context. The two cohorts are the strata.

comment:31 Changed 5 years ago by dconnolly

  • Resolution set to fixed
  • Status changed from accepted to closed

comment:32 Changed 4 years ago by dconnolly

  • Keywords public-web added

comment:33 Changed 4 years ago by kcrane2

Approved for public release.

Note: See TracTickets for help on using tickets.