{"id":861,"date":"2016-05-13T15:20:35","date_gmt":"2016-05-13T19:20:35","guid":{"rendered":"http:\/\/www.chiplynch.com\/wordpress\/?p=861"},"modified":"2016-05-18T12:12:31","modified_gmt":"2016-05-18T16:12:31","slug":"on-repeated-medical-testing","status":"publish","type":"post","link":"https:\/\/www.chiplynch.com\/wordpress\/?p=861","title":{"rendered":"On Repeated Medical “testing”"},"content":{"rendered":"

I’ve blogged about this before<\/a>, but I wanted to revisit because FiveThirtyEight<\/a>, whom I love and adore, just posted about it again<\/a>, in light of the recent Theranos<\/a> problems<\/a>.<\/p>\n

My last blog post was long, so let me simplify my position… there’s a big graphic in the middle of the FiveThirtyEight article:<\/p>\n

\"http:\/\/i2.wp.com\/espnfivethirtyeight.files.wordpress.com\/2016\/05\/hobson-theranos-1-rk.png\"<\/a><\/p>\n

And it’s fine, it’s very simple, very accurate math, EXCEPT, it only deals with the case where each person takes exactly one test. \u00c2\u00a0This isn’t FiveThirtyEight’s fault, they borrowed a common example, and that example is what most medical research is based on… see the flow chart in my last post… you take blood, you look for things, and you (doctors and patients) react to what you find.<\/p>\n

This is NOT the best way forward if cheap reapeatable tests become available. \u00c2\u00a0 It’s not: test, react, test, react, test, react – which I agree leads to over-reactive medicine. \u00c2\u00a0It’s: test, test, test, react, which I’m arguing will reduce over-reaction (but I admit may not be for the feint of heart).<\/p>\n

Here’s how the math works with the same numbers as above if we reduce test costs to 10% and spend the same amount in testing… that is, every person gets 10 tests for the same cost.<\/p>\n

Important note… my probabilities, statistics, and combinatorics are rusty (thus the simulation at the end), but if these numbes are off, they’re not far off:<\/p>\n

We have to assume those tests are independent — that is, something that triggers a false positive in one person isn’t some consistent genetic or chemical issue. \u00c2\u00a0Anyway.<\/p>\n

    \n
  1. For every 1,000 people, 20 people have the disease (still).<\/li>\n
  2. Everyone will test positive between 0 and 10 times (because 10 tests each)\n
      \n
    1. Among those with the disease, the data looks more like a continum than binary (WHICH IS WHAT IT SHOULD LOOKE LIKE)… \u00c2\u00a0what do I mean?\n
        \n
      1. Odds are pretty strong that an individual will have 7,8,9, or 10 positive results out of 10. \u00c2\u00a0The odds of the first being a false negative are 1 in 10 (10% or 0.1). \u00c2\u00a0The odds of the first two being negative are 0.1*0.1, three = 0.1*0.1*0.1, and so forth. \u00c2\u00a0So the odds of someone with the disease having ten negative tests are astronomically low. \u00c2\u00a0Low enough that in the twenty\u00c2\u00a0example people you should never really\u00c2\u00a0expect someone to have less than five\u00c2\u00a0positive results.<\/li>\n<\/ol>\n<\/li>\n
      2. Among those WITHOUT the disease, since the test is more precise, things are even better:\n
          \n
        1. Odds of a single false positive are 0.05, so the odds of 10 false positives for one person are ridiculous (0.05^10 = 9.77*10^-14, right?). \u00c2\u00a0 In a group of 1000 patients it will be surprising to see anyone with more than 4 false positives.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<\/li>\n
        2. At this point, for me, it’s easier to simulate this than to do the math (my math degree is 20 years old after all). \u00c2\u00a0And we can do that quickly in a spreadsheet or an R or python program.\n
            \n
          1. Make 1000 records, and label 20 with “1” (true positive) and the rest with 0 (true negative).<\/li>\n
          2. Assign 10 random percentages\u00c2\u00a0to each record.<\/li>\n
          3. For the 20 true positives, count the number of tests that are below\u00c2\u00a00.9<\/li>\n
          4. For the 980 true negatives, count the number of tests that are above\u00c2\u00a00.95<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

            Here’s what I got:<\/p>\n

            \"Results<\/a>

            Results of simulating 1000 patients and 10 tests each.<\/p><\/div>\n

            Which doesn’t look so bad, does it? \u00c2\u00a0The Green are the healthy patients, who all had less than 5 positive tests, and the strange coral color are the unhealthy patients, who all had more than 5 positive tests. \u00c2\u00a0Yes, with large enough numbers of patients, 10 tests would start to see some anomalies bleed over, but compared to taking a single test and a single result, we’ve gone from a 27% result to a 100% result. \u00c2\u00a0YES, it cost 10 times as much, in testing, but tests are way cheaper than treatment in most situations, and driving test costs down has a much stronger ROI, if we can get a three-ish-fold return in accuracy (27 to 100).<\/p>\n

            If doctors are complaining that the 368 patients who received at least one false positive are going to be an issue (269+64+35), then they need to be shown this chart and explained basic math. \u00c2\u00a0The fact that we make bad medical decisions because people don’t understand the realities of medical tests is NOT a reason to turn down improvments in data collection.<\/p>\n

            ObFinalNote: \u00c2\u00a0Theranos apparently has been, if not falsifying, at least misleading or misrepresenting the\u00c2\u00a0accuracy of its tests. \u00c2\u00a0NO solution exists for tests that simply don’t work. \u00c2\u00a0But given the thought experiment on the table, the math is in FAVOR of more testing, not less, if a) it becomes cost effective and b) we keep our heads about us.<\/p>\n","protected":false},"excerpt":{"rendered":"

            I’ve blogged about this before, but I wanted to revisit because FiveThirtyEight, whom I love and adore, just posted about it again, in light of the recent Theranos problems. My last blog post was long, so let me… Read More<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[30,34],"tags":[],"class_list":["post-861","post","type-post","status-publish","format-standard","hentry","category-healthcare","category-rant"],"_links":{"self":[{"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/861","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=861"}],"version-history":[{"count":5,"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/861\/revisions"}],"predecessor-version":[{"id":871,"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/861\/revisions\/871"}],"wp:attachment":[{"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=861"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=861"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.chiplynch.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=861"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}