| 1 | [[PageOutline]] |
| 2 | |
| 3 | nearby: DevTools |
| 4 | |
| 5 | While we are [[Using I2B2]] for the bulk of the functionality of [[HERON]], |
| 6 | [source:raven-j heron-admin] provides a regulatory enforcement web layer for [[HERON#governance]]. [[HERONTrainingMaterials]] provides end-user documentation. (''Well, it used to; see #1268) |
| 7 | |
| 8 | ''in progress; see ticket:600#comment:11 for context'' |
| 9 | |
| 10 | ''Automated tests to match [[HERONTrainingMaterials]] sure would be nice.'' |
| 11 | |
| 12 | == Design == |
| 13 | |
| 14 | Developer documentation, including doctests, is generated from the code: |
| 15 | |
| 16 | * [http://informatics.kumc.edu/devdoc/heron-admin/ heron_wsgi: Policy Enforcement Web Interface for HERON] |
| 17 | |
| 18 | === Access Policy and Authority Flow === |
| 19 | |
| 20 | Anyone who can login (via CAS) can review usage stats or sign a system access agreement (which is recorded in [[REDCap]]). |
| 21 | |
| 22 | Investigators, i.e. faculty and executives, can submit investigator requests (sponsorship, data use) to the DROC. |
| 23 | |
| 24 | {{{ |
| 25 | #!graphviz |
| 26 | |
| 27 | strict digraph "policy" { |
| 28 | graph [ |
| 29 | rankdir = "TB", |
| 30 | ] |
| 31 | |
| 32 | Config -> REDCap_rd |
| 33 | Config -> REDCapAPI |
| 34 | Config -> LDAP_rd |
| 35 | Config -> jboss_deploy |
| 36 | jboss_deploy -> i2b2pm_rd |
| 37 | |
| 38 | login [label="CAS login"] |
| 39 | |
| 40 | login -> Faculty |
| 41 | login -> Executive |
| 42 | login -> Sign |
| 43 | Sign [color=green, label="Sign SAA"] |
| 44 | REDCapAPI -> Sign |
| 45 | |
| 46 | |
| 47 | Config -> Executive |
| 48 | REDCap_rd -> DROC |
| 49 | login -> DROC |
| 50 | |
| 51 | DROC -> Audit |
| 52 | i2b2pm_rd -> Audit |
| 53 | |
| 54 | Audit [color=green] |
| 55 | |
| 56 | LDAP_rd -> Faculty |
| 57 | |
| 58 | Executive -> inv |
| 59 | Faculty -> inv |
| 60 | |
| 61 | inv [shape=invhouse, label="any"] |
| 62 | |
| 63 | inv -> invreq |
| 64 | inv -> build_team |
| 65 | |
| 66 | LDAP_rd -> build_team |
| 67 | build_team [color=green] |
| 68 | |
| 69 | invreq [color=green, label="Investigator Request"] |
| 70 | |
| 71 | REDCapAPI -> invreq |
| 72 | |
| 73 | login -> Usage |
| 74 | |
| 75 | i2b2pm_rd -> Usage |
| 76 | Usage [ color=green] |
| 77 | } |
| 78 | |
| 79 | }}} |
| 80 | |
| 81 | Investigators and sponsored users, provided their human subjects training is current and they have signed the system access agreement, can follow the "start i2b2 query tool" link; if they have an acknowledgement on file, they can use i2b2; otherwise, they can acknowledge the disclaimer. |
| 82 | |
| 83 | {{{ |
| 84 | #!graphviz |
| 85 | |
| 86 | strict digraph "policy" { |
| 87 | graph [ |
| 88 | rankdir = "TB", |
| 89 | ] |
| 90 | |
| 91 | Config -> REDCap_rd |
| 92 | Config -> REDCapAPI |
| 93 | Config -> LDAP_rd |
| 94 | Config -> jboss_deploy |
| 95 | jboss_deploy -> i2b2pm_wr |
| 96 | Config -> Chalk |
| 97 | |
| 98 | login [label="CAS login"] |
| 99 | |
| 100 | |
| 101 | login -> Faculty |
| 102 | login -> Executive |
| 103 | login -> Sponsored |
| 104 | login -> Training |
| 105 | login -> SAA |
| 106 | login -> Acknowledgement |
| 107 | |
| 108 | |
| 109 | |
| 110 | REDCap_rd -> SAA |
| 111 | REDCap_rd -> Sponsored |
| 112 | REDCap_rd -> Acknowledgement |
| 113 | |
| 114 | Config -> Executive |
| 115 | |
| 116 | Chalk -> Training |
| 117 | |
| 118 | LDAP_rd -> Faculty |
| 119 | |
| 120 | Faculty -> qualified |
| 121 | Executive -> qualified |
| 122 | Sponsored -> qualified |
| 123 | |
| 124 | qualified [shape=invhouse, label="any"] |
| 125 | |
| 126 | qualified -> start |
| 127 | Training -> start |
| 128 | SAA -> start |
| 129 | |
| 130 | start [shape=invtriangle, label="all"] |
| 131 | |
| 132 | REDCap_rd -> Disclaimer |
| 133 | Disclaimer -> Ack |
| 134 | |
| 135 | start -> Ack |
| 136 | REDCapAPI -> Ack |
| 137 | Ack [color=green] |
| 138 | |
| 139 | start -> access |
| 140 | Acknowledgement -> access |
| 141 | access [shape=invtriangle, label="all"] |
| 142 | |
| 143 | access -> i2b2_authz |
| 144 | |
| 145 | i2b2_authz [color=green] |
| 146 | |
| 147 | i2b2pm_wr -> i2b2_authz |
| 148 | |
| 149 | } |
| 150 | |
| 151 | }}} |
| 152 | |
| 153 | |
| 154 | === Module Dependencies === |
| 155 | |
| 156 | {{{ |
| 157 | #!graphviz |
| 158 | |
| 159 | |
| 160 | # This file was generated by sfood-graph. |
| 161 | # and then the boring stuff was edited out |
| 162 | |
| 163 | strict digraph "dependencies" { |
| 164 | graph [ |
| 165 | rankdir = "LR", |
| 166 | overlap = "scale", |
| 167 | size = "8,10", |
| 168 | ratio = "fill", |
| 169 | fontsize = "16", |
| 170 | fontname = "Helvetica", |
| 171 | clusterrank = "local" |
| 172 | ] |
| 173 | |
| 174 | node [ |
| 175 | fontsize=7 |
| 176 | shape=ellipse |
| 177 | // style=filled |
| 178 | // shape=box |
| 179 | ]; |
| 180 | |
| 181 | // node [ |
| 182 | // fontsize=7 |
| 183 | // style=ellipse |
| 184 | // ]; |
| 185 | |
| 186 | "heron_wsgi/admin_lib/checklist.py" [style=filled]; |
| 187 | "heron_wsgi/admin_lib/checklist.py" -> "heron_wsgi/admin_lib/heron_policy.py"; |
| 188 | "heron_wsgi/admin_lib/checklist.py" -> "heron_wsgi/admin_lib/medcenter.py"; |
| 189 | "heron_wsgi/admin_lib/disclaimer.py" [style=filled]; |
| 190 | "heron_wsgi/admin_lib/disclaimer.py" -> "heron_wsgi/admin_lib/redcap_connect.py"; |
| 191 | "heron_wsgi/admin_lib/disclaimer.py" -> "heron_wsgi/admin_lib/redcapdb.py"; |
| 192 | "heron_wsgi/admin_lib/heron_policy.py" [style=filled]; |
| 193 | "heron_wsgi/admin_lib/heron_policy.py" -> "heron_wsgi/admin_lib/disclaimer.py"; |
| 194 | "heron_wsgi/admin_lib/heron_policy.py" -> "heron_wsgi/admin_lib/i2b2pm.py"; |
| 195 | "heron_wsgi/admin_lib/heron_policy.py" -> "heron_wsgi/admin_lib/medcenter.py"; |
| 196 | "heron_wsgi/admin_lib/heron_policy.py" -> "heron_wsgi/admin_lib/noticelog.py"; |
| 197 | "heron_wsgi/admin_lib/heron_policy.py" -> "heron_wsgi/admin_lib/redcap_connect.py"; |
| 198 | "heron_wsgi/admin_lib/heron_policy.py" -> "heron_wsgi/admin_lib/redcapdb.py"; |
| 199 | "heron_wsgi/admin_lib/i2b2pm.py" [style=filled]; |
| 200 | "heron_wsgi/admin_lib/medcenter.py" [style=filled]; |
| 201 | "heron_wsgi/admin_lib/noticelog.py" [style=filled]; |
| 202 | "heron_wsgi/admin_lib/noticelog.py" -> "heron_wsgi/admin_lib/redcapdb.py"; |
| 203 | "heron_wsgi/admin_lib/redcap_connect.py" [style=filled]; |
| 204 | "heron_wsgi/admin_lib/redcapdb.py" [style=filled]; |
| 205 | "heron_wsgi/cas_auth.py" [style=filled]; |
| 206 | "heron_wsgi/drocnotice.py" [style=filled]; |
| 207 | "heron_wsgi/drocnotice.py" -> "heron_wsgi/admin_lib/heron_policy.py"; |
| 208 | "heron_wsgi/drocnotice.py" -> "heron_wsgi/admin_lib/noticelog.py"; |
| 209 | "heron_wsgi/drocnotice.py" -> "heron_wsgi/admin_lib/redcapdb.py"; |
| 210 | "heron_wsgi/drocnotice.py" -> "heron_wsgi/genshi_render.py"; |
| 211 | "heron_wsgi/genshi_render.py" [style=filled]; |
| 212 | "heron_wsgi/heron_srv.py" [style=filled]; |
| 213 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/admin_lib/checklist.py"; |
| 214 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/admin_lib/disclaimer.py"; |
| 215 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/admin_lib/heron_policy.py"; |
| 216 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/admin_lib/medcenter.py"; |
| 217 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/admin_lib/redcap_connect.py"; |
| 218 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/admin_lib/redcapdb.py"; |
| 219 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/cas_auth.py"; |
| 220 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/drocnotice.py"; |
| 221 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/genshi_render.py"; |
| 222 | "heron_wsgi/heron_srv.py" -> "heron_wsgi/stats.py"; |
| 223 | "heron_wsgi/stats.py" [style=filled]; |
| 224 | "heron_wsgi/stats.py" -> "heron_wsgi/admin_lib/heron_policy.py"; |
| 225 | |
| 226 | |
| 227 | } |
| 228 | |
| 229 | |
| 230 | }}} |
| 231 | |
| 232 | == Development process #dev-process |
| 233 | |
| 234 | [http://genshi.edgewall.org/wiki/Documentation/xml-templates.html genshi template language] @@... |
| 235 | |
| 236 | see source:raven-j/heron_wsgi/Makefile and |
| 237 | source:raven-j/README.rst |
| 238 | |
| 239 | |
| 240 | === Clone Repository |
| 241 | |
| 242 | UsingVersionControl: Clone the [https://bmi-work.kumc.edu/changes/raven-j raven-j] (a.k.a Heron Admin) repository from using your preferred method (e.g. TortoiseHg, Hg command-line, etc.). |
| 243 | |
| 244 | === 2. Setup Virtualenv === |
| 245 | |
| 246 | If you want use anaconda as virtualenv, please use the following pattern. |
| 247 | Please push requirements.sh and requirements.txt to a git repo. |
| 248 | |
| 249 | {{{ |
| 250 | cat requirements.sh |
| 251 | ############################################################################################ |
| 252 | # Part-1 How did you create the environment (this helps in future if you need to recreate it) |
| 253 | ############################################################################################ |
| 254 | |
| 255 | # conda create -n lav_test_env python=2.7 |
| 256 | # source activate lav_test_env |
| 257 | |
| 258 | # Now installing pip inside virtul-env |
| 259 | # conda install pip |
| 260 | |
| 261 | # I will manually install the latest version of pandas, during this installation, I will |
| 262 | # find out what is the version(by pip freeze), and then I 'll put it in requiremnts.txt. |
| 263 | # pip install pandas |
| 264 | |
| 265 | # if you know the exact version, skip the above step and write the exact version in requirements.txt |
| 266 | # Please always write versions for each libary in requiremnts.txt |
| 267 | # pip install -r requiremnts.txt |
| 268 | |
| 269 | # if you can't install the certain library with pip, then install with conda |
| 270 | # conda install cx_Oracle |
| 271 | |
| 272 | ############################################################################################ |
| 273 | # Part-2 activate env |
| 274 | ############################################################################################ |
| 275 | |
| 276 | source activate lav_test_env |
| 277 | |
| 278 | ############################################################################################ |
| 279 | # Part-3 Print dependencies as well as child dependencies |
| 280 | ############################################################################################ |
| 281 | |
| 282 | which python |
| 283 | python --version |
| 284 | pip freeze |
| 285 | conda env export |
| 286 | |
| 287 | ############################################################################################ |
| 288 | # Part-4 Run Code |
| 289 | ############################################################################################ |
| 290 | |
| 291 | python test.py |
| 292 | |
| 293 | ############################################################################################ |
| 294 | # Part-5 deactivate env |
| 295 | ############################################################################################ |
| 296 | |
| 297 | source deactivate |
| 298 | }}} |
| 299 | |
| 300 | dependencies... mysql, cx_Oracle, etc. ... PythonVirtualEnvironment, PythonOnWindows |
| 301 | |
| 302 | 2.1. Create an associated Python virtualenv (optimally separated from the code). |
| 303 | {{{ |
| 304 | $ virtualenv <venv-path> |
| 305 | }}} |
| 306 | 2.2. Activate your virtualenv. |
| 307 | {{{ |
| 308 | $ source <venv-path>/bin/activate |
| 309 | }}} |
| 310 | 2.3. Use PIP to install necessary Python |
| 311 | {{{ |
| 312 | $ cd <raven-j-path> |
| 313 | $ pip install -r requirements.txt |
| 314 | }}} |
| 315 | ... and (introduced in #653 `ldap_cert_653`; TODO: merge requirements files) ... |
| 316 | {{{ |
| 317 | $ cd <raven-j-path>/traincheck/ |
| 318 | $ pip install -r requirements.txt |
| 319 | }}} |
| 320 | |
| 321 | ''see also PythonVirtualEnvironment, PythonOnWindows, WritingQualityCode#pypy)'' |
| 322 | |
| 323 | Unit test can be run with: |
| 324 | {{{ |
| 325 | $ python -m doctest <file> |
| 326 | }}} |
| 327 | |
| 328 | == CLI Integration Testing == |
| 329 | |
| 330 | see GroupOnly/HeronAdmin |
| 331 | |
| 332 | |
| 333 | == History == |
| 334 | |
| 335 | |
| 336 | oops... heron-admin is now ambiguous: support email address and software package. |
| 337 | |
| 338 | |
| 339 | * milestone:RavenBeta, milestone:heron-system-access for development history |