Changes between Initial Version and Version 59 of DevTools


Ignore:
Timestamp:
05/18/15 09:46:52 (3 years ago)
Author:
dconnolly
Comment:

update XP planning game link

Legend:

Unmodified
Added
Removed
Modified
  • DevTools

    v1 v59  
     1[[PageOutline]]
     2
     3== Development Process == #process
     4
     5  At first the tickets were just a way for me to track my own work, but we're getting pretty good at using trac for an approximation of the [http://www.extremeprogramming.org/rules/planninggame.html XP planning game]: Russ plays business customer and I represent development. Sometimes sketching milestones and breaking them down into tickets is straightforward enough, but sometimes I don't understand what he's asking for, so I ask him to tell me stories about how it works, and we capture those in the wiki.
     6
     7  -- [http://www.madmode.com/2010/08/setting-up-project-tracking-with-trac.html Setting up project tracking with trac, hg, and a little help from my friends on freenode], Dan C., Aug 2010
     8
     9See also:
     10 - UsingVersionControl, especially setting up TortoiseHG on Windows
     11 - HeronProjectTimeline (whose name is a bit ironic, since the trac uses "timeline" to look back; for looking forward, it uses "roadmap")
     12 -  UnderTheHood - details about this project management and issue tracking system
     13
     14=== Testing, documentation, and quality software development === #quality
     15
     16[=#doctest]See WritingQualityCode.
     17
     18== Technical Topics == #tech
     19
     20=== Databases, Data Warehousing, and Information Security === #db
     21
     22    * SQL - central to just about everything we do: HeronLoad, [[REDCap]], etc.
     23      - [http://www.w3schools.com/sql/trysql.asp?filename=trysql_select_all try SQL] at W3Schools
     24      - [http://sqlitebrowser.org/ DB Browser] for [http://en.wikipedia.org/wiki/SQLite SQLite]
     25      - [http://en.wikipedia.org/wiki/SQL SQL] and [http://en.wikipedia.org/wiki/Relational_algebra relational algebra] in wikipedia
     26      - Russ has some books
     27    * OracleTips
     28      * TunnelingOracleConnections to "encrypt data in motion"
     29      * [http://www.oracle-base.com/articles/misc/killing-oracle-sessions.php Ways to kill your session ]
     30      * [http://guyharrison.squarespace.com/opsgsamples/ Tuning scripts]
     31      * [http://www.dbspecialists.com/files/presentations/temp_space.html Help with Temporary Tablespace Issues]
     32      * [http://oradbaeasy.wordpress.com/2009/06/11/importing-and-exporting-oracle-database-using-datapump-utility/ How datapump exports work]
     33    * Data Warehousing (including ETL)
     34      - [http://en.wikipedia.org/wiki/Star_schema Star schema (wikipedia)] is a central concept in, e.g., our HeronLoad ETL work.
     35        - The Muphy et. al. paper in [[Using I2B2]] shows the I2B2 star schema.
     36      - [http://en.wikipedia.org/wiki/Ralph_Kimball Kimball] "wrote the book," or books; Russ has them.
     37      - ''Pentaho Solutions'' covers an emerging open source approach (see also: notes in ticket:1)
     38    * [[HERONsecuritynotes]]
     39
     40=== Web Design, Security === #web-design
     41[[HERON]], [[REDCap]], and [[CRIS]] all use the Web for user interface.
     42
     43   * [http://oreilly.com/catalog/9781934356135/ Web Design for Developers] - Dan got it based on a [http://favstar.fm/users/timbray/status/6903704865 nod from Tim Bray]. ''I haven't read it yet, though''
     44   * ''Tangled Web'' book on web security
     45   * AuthorityInjection -- patterns of secure composition
     46   * HeronAdminDev
     47   * UsingDrupal
     48   * We've had good experience with the python pyramid web framework; see [9e4f4c169c5e/reqroute] to get started. It's known for a "pay only for what you use" style, and it works reasonably well with AuthorityInjection, since it doesn't rely on globals.
     49
     50If you're new to HTML, HTTP, CSS, etc., see [http://www.w3.org/community/webed/wiki/How_does_the_Internet_work How does the Internet Work] etc. in the [http://www.w3.org/community/webed/wiki/Main_Page#Web_Standards_Curriculum_table_of_contents Web Standards Curriculum].
     51
     52==== Opening New Browser Windows ====
     53
     54Opening New Browser Windows is one of the [http://www.useit.com/alertbox/9605.html top 10 mistakes in web design], according to Jacob Nielsen. It seems to be a KUMC norm, meanwhile. ''todo (Dan): find KUMC web styleguide and see if this is a conscious decision, and what it would take to get it changed.''
     55
     56
     57=== Programming Languages: Python, Javascript, PHP, Java === #prog
     58
     59See WritingQualityCode to get started with our emerging (#1046) preferred toolset that supports all of these (as well as SQL and HTML):
     60
     61   * python - we use it in our HeronLoad ETL process, HeronAdminDev, web integration, etc.
     62      - [http://coffeeghost.net/2008/08/30/python-cheat-sheet/ python cheat sheet]
     63      * [http://www.python.org/dev/peps/pep-0008/ PEP: 8 Style Guide for Python Code]
     64      - [http://docs.python.org/tutorial/ python tutorial]
     65        - ''Python is a language you can get into on one battery!'' -- [http://www.w3.org/2000/10/swap/#L88 Tim Berners-Lee, Oct 2010]
     66      - [http://www.python.org/doc/ official python docs] are a great reference. (multiple formats are available, including typeset versions for printing.)
     67      - Russ has some books
     68      - deployment: PythonOnWindows, PythonVirtualEnvironment
     69   * [=#js Javascript] - We're [[Using I2B2]], which has a javascript front-end.
     70      * JavaScriptTips ''especially The Good Parts, by Crockford''
     71   * PHP
     72     * We are UsingDrupal for [[CTSA#frontiers-web]] and we use [[REDCap]]; both are written in PHP.
     73     * Tamara has a big book
     74     * [http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/ PHP: a fractal of bad design], Apr 2012 by eevee, explains pretty well why Dan avoids it, as a rule.
     75   * Java
     76     * We're not doing much Java development, but we're [[Using I2B2]], which has a middle tier written in Java.
     77     * JavaDevTools
     78     * ''Effective Java'' 2nd Ed by Joshua Bloch
     79
     80[http://stackoverflow.com/ Stack Overflow] is a great community resource for all of the above (and much of the below...).
     81
     82[http://software-carpentry.org/4_0/ Software Carpentry] is a good source for somebody who's new to programming.
     83
     84=== System administration, web server administration === #sysadmin
     85   * InformationResources - our partners in monitoring, administration of [[HERON]],  [[REDCap]], [[CRIS]], etc.
     86     * InformationResources#ApacheSecurity
     87     * [wiki:InformationResources#suse-cve SuSE CVS Security Index]
     88   * ApacheTroubleShooting
     89   * [query:keywords=~ops ops keyword tickets]
     90
     91==== Amazon Web Services ==== #AWS
     92
     93   * AmazonWebServices
     94
     95=== Statistics and Data Analysis with R === #stats
     96
     97Our work increasingly integrates stats, e.g. HeronStatsPlugins.
     98
     99  * [http://www.rstudio.com/ RStudio] - desktop, web tool for R development
     100  * [http://stat405.had.co.nz/ Introduction to data analysis] Hadley Wickham Fall 2012. Rice University
     101  * [=#R R]: [http://www.r-project.org/ R project]
     102     * [http://cran.r-project.org/doc/manuals/R-intro.html An Introduction to R]
     103     * [http://cran.r-project.org/doc/manuals/R-lang.html R Language Definition]
     104  * ''Introductory Statistics with R'' by Peter Dalgaard (Russ's)
     105  * [http://google-styleguide.googlecode.com/svn/trunk/google-r-style.html Google's R Style Guide]
     106  * [http://adv-r.had.co.nz/ Advanced R programming] by Hadley Wickham
     107  * [http://www.diigo.com/user/dckc-madmode/Statistics%2Cprogramming Dan's bookmarks tagged statistics and programming]
     108  * Dan got ''The Art of R Programming. '' on the basis of  recommendations such as [http://stackoverflow.com/questions/192369/books-for-learning-the-r-language Nov 2011 on Stack Overflow]
     109     - It has, for example, **nothing on automated testing**; so I'm not sure I'd recommend it.
     110
     111''WritingQualityCode#r-pkg is starting to take shape.''
     112
     113== Coding chunks and Collaboration Mechanics == #coding-chunks
     114
     115  I maintain that programming cannot be done in less than three-hour windows.  It takes three hours to spin up to speed, gather your concentration, shift into "right brain mode", and really focus on a problem.  Effective programmers organize their day to have at least one three-hour window, and hopefully two or three.
     116  -- [http://www.w-uh.com/articles/030308-tyranny_of_email.html The Tyranny of Email], Ole Eichhorn, March 2003
     117
     118''I disagree with Eichhorn about [ticket:160 shared text chat]. More on that below. -- Dan''
     119
     120=== Meetings ===
     121
     122Meetings are valuable, but costly too. So let's be judicious about doing them, and do them right. (Avoid the [http://www.fastcompany.com/magazine/02/meetings.html The Seven Sins of Deadly Meetings])
     123
     124  One reason programmers dislike meetings so much is that they're on a different type of schedule from other people. Meetings cost them more. -- [http://www.paulgraham.com/makersschedule.html Maker's Schedule, Manager's Schedule], Paul Graham, July 2009
     125
     126 * CodeRevewNotes (ugh... misspelled... need trac wiki rename tool)
     127
     128=== Improving efficiency with text chat === #group-chat
     129
     130''See also #160 on shared text chat, GPC:ticket:46 for gpc-dev chat, and ticket:751#comment:2 for a note on Adobe Connect.''
     131
     132We (Biomedical Informatics) are physically co-located, so it's less
     133critical than when collaborating remotely, but it's still useful:
     134
     135 * technical details are often easier to communicate in text than by voice (e.g. error messages, URLs, commands to type)
     136 * text chat is near-real-time; you can think out loud and somebody might respond right away or quite a while later or not at all.
     137    The random rewards of gambling are much more seductive than a more predictable reward cycle.
     138    -- [http://www.boston.com/news/globe/ideas/articles/2007/08/19/your_brain_on_gambling/ Your brain on gambling]
     139 * People shouldn't be expected to respond to email instantly. Allowing email to interrupt work is terribly counter-productive; it's best to leave it be until a natural stopping point.
     140 * Ringing somebody's phone is sure to break their concentration.
     141
     142The interruption costs more than just the minute or two to deal with the incoming call or message:
     143
     144  It typically takes about 15 minutes of uninterrupted study to get into a state of "flow", and the constant interruptions and distractions of a typical office environment will force you out of "flow" and make productivity impossible to achieve.
     145
     146  In your development process, identify the activities that disrupt flow, and modify them until they don't. ... For example, consider getting rid of the telephone.
     147
     148  -- [http://c2.com/cgi/wiki?MentalStateCalledFlow Mental State Called Flow] in the [http://c2.com/cgi/wiki original wiki on People Projects And Patterns in Software Development].
     149
     150With shared text chat, people can choose how much attention to give it any time, to participate or not, respond quickly or slowly, etc.
     151This leads to some suggestions:
     152
     153 * Rather than asking "may I interrupt you?" face to face or by phone, consider text chat first. There's no way to say "no" in the other two media.
     154 * If it's important but not urgent, use email.
     155 * Feel free to turn off or tune out text chat; if it's critical, people can escalate to other means.
     156
     157For real-time collaboration with (and support from) the wider open source community, see FreeNode.
     158
     159See also [http://www.advogato.org/person/connolly/diary/50.html Social standards and coding fugues], Dan C., Feb 2007
     160
     161[https://pidgin.im/ Pidgin] is a cross-platform (Windows and Linux at least) chat client that supports IRC, XMPP (like Google Hangouts) and many others.  One nice feature is that you can configure the "Message Notification" plugin (shipped with the Windows version of Pidgin) to only flash the icon to grab your attention if someone mentions your name in the chat window.
     162 * Tools -> Plugins -> "Message Notification": Tick the box for "enabled" then select "Configure Plugin".
     163
     164== Recruiting, Interviewing ==
     165   * [http://www.amazon.com/Cracking-Coding-Interview-Programming-Questions/dp/1466208686/ref=zg_bs_5_20?_encoding=UTF8&zgs=books&zgrt=top-sellers&zgid=5 Cracking the Coding Interview: 150 Programming Questions and Solutions] by Gayle Laakmann