Talk:WikiJournal Preprints/Comparison between the Lund-Browder chart and the BurnCase 3D® for consistency in estimating total body surface area burned

From Wikiversity
Jump to navigation Jump to search

WikiJournal Preprints logo.svg

WikiJournal Preprints
Open access • Publication charge free • Public peer review

WikiJournal User Group is a publishing group of open-access, free-to-publish, Wikipedia-integrated academic journals. <seo title=" Wikiversity Journal User Group, WikiJournal Free to publish, Open access, Open-access, Non-profit, online journal, Public peer review "/>

<meta name='citation_doi' value=>

Article information

Authors: Kyung Tak Yoo ORCID iD.svg , Go Woon Woo, Tae Young Jang, Jae Seok Song[a][i]

Yoo, K; Woo, G; Jang, T; Song, J. 




Copyvio check[edit]

Initial editorial comments[edit]

Comments by Thomas Shafee
These comments were submitted on , and refer to this previous version of the article

As a minor edit, I would recommend omitting the "®" symbol after the first use, since it is thereafter implied. The layout, format, and linking are all appropriate.

Response

Thank you so much for reviewing the manuscript. The following is my responses and revised text reflecting your comments.

As per your comment on ‘registered mark’, the expression ‘registered mark’ has been removed except in the first occurrences in the abstract and main article. I regret that we are unable to modify the title itself.
--Stonegaze (discusscontribs) 04:47, 26 November 2018 (UTC)

First peer reviewer[edit]

Review by anonymous peer reviewer , MD PhD burns researcher
This review was submitted on , and refers to this previous version of the article


This is an interesting study and an important one. There are however several concerns that need to be addressed.

First, 3D image is not novel and similar type of studies have been conducted before. what is new and different about this study?

Second, you only have small burns and i think this is a bias introduced in this study. smaller burns are easier to estimate than burns over 20% tbsa. you need to add some larger burn sizes.

Third, we are aware of the challenge to estimate burn size in adipose patients. how was the BMI of your subjects?

Fourth, i am concerned about only 3 surgeons. again, why only 3 and not more?

Fifth, there are many limitations that require mentioning but the main one is what is novel and different compared to other already published trials.

Response

Thank you so much for reviewing the manuscript. The following is my responses and revised text reflecting your comments.

Answer about first and fifth issues : identical difference between this article and other researches

As you pointed out, TBSA (burn size estimation) using computers is an active research topic globally. However, to the best of our knowledge, we have found no paper written on the topic by South Korean scholars. One motivation for our study was that this paper, essentially as a preliminary study, would hopefully trigger subsequent research activities on estimating burn area using computers. The revised portion of the paper is as following:


A sentence at the last part of introduction section was fixed :

“...In this paper, we compare the difference of the estimation for %TBSA produced with the BurnCase 3D and the conventional Lund-Browder chart as observed on burn patients’ pictures, as a preliminary study, this paper investigates pros and cons of adoption."


A new paragraph was added in Discussion section :

Despite sufficiently wide spread of EMR (electronic medical record) systems South Koreain South Korea, little work has been published on computer-aided burn size estimation. In writing this paper as a preliminary study, we hoped to trigger future research activities. We did not include statistically sufficient number of study participants or patients, however, we were able to demonstrate differences of provisional vs. computer-aided burn area estimation methods.


A first sentence of conclusion section was changed :

As a preliminary study, this paper observed some variance in %TBSA estimation among participants when using the Lund-Browder chart and when using the BurnCase 3D.


Answer to the second issue : need more subjects which are larger than 20% TBSA

Thank you for this important comment. Between 2013 and 2017, the number of burn patients admitted to one of the author’s hospital was 52 and a total of 2,244 photos have been taken on the patients. However, we ended up with only 263 photos from 26 patients for this paper after discarding pictures that could not be used for various factors: burn size/location could not be determined; presence of blister or slough made estimation impossible; photos were taken after surgery. We agree that the numbers are too small for making reliable claims, however, our consultation with a statistician indicated the numbers are large enough for conducting a preliminary study. Even so, we very strongly agree with you on the point that we need to collect more photos from more patients.


Answer to the third issue : comment about BMI of subjects

In the chart we provided BMI data yet we omitted explanation on it in the text; the following has been added to the Result section.


Additional descriptions were added after the third sentence of Result section :

"...and their body mass index was 24.6±3.4kg/m2. World Health Organization(WHO) defines 25kg/m2 and 30kg/m2 in BMI as overweight and obesity each in general. However, Korean Society for the Study of Obesity(KOSSO) and Asia-Pacific section of WHO have agreed to use lower indices of 23kg/m2 and 25kg/m2 for overweight and obesity because the prevalence of diabetes and cardiovascular diseases doubles above the 25kg/m2 [1][2][3]. With the regular WHO standard, 13 patients, or 50% of the 26, were overweight and none were in the category of obesity. With the WHO standard for Asians, then, 6 patients (23.1%) were overweight and 13 patients (50%) obese.

  1. Bray, GA; Bouchard, C. (2004). Handbook of obesity. New York, NY: Marcel Dekker, Inc.
  2. WHO (1997). Obesity: Preventing and managing the global epidemic. Report on a WHO consultation on obesity.
  3. WHO/IASO/IOTF (2000). The Asia-Pacific perspective: redefining obesity and its treatment. Health communications Australia: Melbourne.


Answer to fourth issue : Lack of the number of observers

Thank you for this important comment. From the early stage of our study, this concern forced us consult a statistician. We settled on 3 observers in order to reduce inter-observer variation. We revised the manuscript by adding a Material and methods on inter-observer variation.


"... poor quality were discarded. After consulting a statistician, the number of participant was determined to 3 in order to reduce inter-observer variation.”

Other issue

The reason why Participant No. 3’s evaluation results deviate from the other participants is that he was going through the learning curve for the software along the experiment. We revised the manuscript to include this factor.


The fifth sentence at 3rd paragraph in discussion section is changed

from “This hypothesis merits additional research with a large number of participants”

toWe believe discussing the learning curve for BurnCase 3D will merit additional research”.

--Stonegaze (discusscontribs) 05:15, 26 November 2018 (UTC)

Second peer review[edit]

Review by Herbert Haller , HLMedConsult
This review was submitted on , and refers to this previous version of the article


Competing interests declared: medical advisor to the BurnCase 3D project without financial benefits

This paper compares computer-assisted TBSA calculation using BurnCase3D with manual estimation using Lundt-Browder charts. The study’s main findings include significant differences in TBSA estimates between two methods in two out of three participants. No such differences were found in the third participant, who was an experienced BurnCase3D user. The study also found differences in time requirements, with estimation using BurnCase3D taking significantly more time.

As a medical advisor to the BurnCase project, I am happy to see these questions receiving research attention. The study seems to provide evidence that the use of BurnCase can lead to improved accuracy in TBSA estimates at the cost of increased time requirements. Unfortunately, I find two methodological weak points that may lead the authors to underestimate the advantage of computer-assisted methods.

The first weak point is that participants were trained on the use of BurnCase3D, including the use of 5 tutorial cases, before doing any other estimation. It would have been preferable to train participants after they completed estimates using Lundt-Browder since we have found that use of BurnCase3D leads to “calibration effect” which increases accuracy in non-computer assisted estimation methods. Some evidence for this effect is provided by the paper itself which showed that the experienced BurnCase3D user among the three participants showed no differences in TBSA estimates when using BurnCase3D and when using Lundt-Browder. I believe that this leads to the authors underestimating variance between estimates completed using the two methods. It would encourage discussion of this point and it would be interesting to see whether in fact there is evidence of increasing calibration in participants across the different example cases.

The second weak point, which may be methodological or simply a lack of discussion, is regarding the timing estimates for the use of BurnCase3D. There are two ways of using BurnCase3D. The first involves manually fitting a photograph to a 3D model in the application, which takes a lot of time but allows for very precise marking of burns on the 3D model. The second method involves letting users simply mark the burned surface area on the 3D model with reference to the photos, but without explicitly fitting the photos to the 3D model. This second method is much faster and we have found that it is nearly as accurate as the first. Experience shows that clinical practitioners eventually adopt the second method.

Based on the large time requirements reported in the study, I would expect that a mix of these two techniques was used. I think this leads to an overestimation of the effective time requirements for BurnCase3D in clinical practice. Ideally, I would have liked to see a comparison of the two methods of using BurnCase3D as well, but at the very least, the paper in its current state is missing an in-depth discussion of the precise technique used and how it relates to clinical practice. This is especially important since the paper mentions that it aims at analyzing the pros and cons of using computer-assisted methods in practice.

I’d also have been interested to see whether estimates differ based on wound location since one would expect that 3D measurements are superior to 2D methods in the case of lateral wounds. I would also have liked a discussion of some of the large differences in estimates between Lundt-Browder and BurnCase3D in cases where TBSA is large (e.g. >20%). While the paper mentions that there is “some difference” in estimates between the two methods, the rather extreme magnitude of those differences in cases of high TBSA seems worthy of separate mention.

Response

Thank you for this insightful comment. We have revised the manuscript reflecting your words.


Response to issue No. 1: TBSA estimation discrepancy in BurnCase 3D

Thanks to your insightful comment, we realized we should have paid more attention to this issue during our research and consequently we looked into the literature. We could not find studies that squarely addressed this specific question, however, a few works we reviewed after your comment were relevant to this issue to varying degrees, which we included to our discussions section as following:


After the sentence “Participant 3 is an author of this study and had some experience of using the software, which may explain the insignificant statistical difference.”

There is a possibility that in the case of Participant 3, such previous experiences could have improved the burn size estimation. In other words, the experience of using BurnCase 3D and checking the output may have enabled Participant 3 to ‘calibrate’ the result for when not using BurnCase3D. This suggests that results obtained using BurnCase3D are more positive and closer to being objective when compared with classical methods[5][20][21]. This implies that the discrepancy of burn-size estimation could be reduced among burn specialist, medical students, and research associates if BurnCase3D is widely used[5]. However, we believe research on this calibration effect and the learning curve for BurnCase 3D merits additional research."


We also deleted the sentence “Ruling out Participant 3, we could hypothesize that there is a meaningful difference in burn estimation between the Lund-Browder chart method and the BurnCase3D” and “We believe discussing the learning curve for BurnCase 3D will merit additional research“.


Response to Issue No. 2: area estimation via BurnCase 3D

As you pointed out, in BurnCase 3D there are two methods of displaying patient's wounds on the 3D model using photos. We have included this point in the manuscript and revised it as following:


"BurnCase 3D has two methods of estimating %TBSA based on patient’s photos - one that involves superimposing the photo on the 3D model and the other that does not. In the non-superimposition method, the wound and the 3D model are merely displayed separately. Superimposing photos on the 3D model creates a more accurate wound representation at the cost of more estimation time by users. On the other hand, without superimposition, %TBSA estimation can be performed faster and the difference is very marked when the burn area is large." Participants spent around 2.5 minutes for burn estimation with the Lund-Browder chart and more than 9 minutes with the BurnCase 3D, and more than 10 minutes for cases when %TBSA was thought to be greater than 10%. The greater the number of burn images, the more time our participants took for estimation. For our experiment, all of the photos were superimposed, and this may have contributed to the increased estimation time for each participant. Had we provided only the non-superimposition method, their estimation time may have been reduced, however, this factor was not anticipated in the experiment designing phase. In our future work, we will adopt both methods for %TBSA estimation."


We also deleted the sentence “We did not perform a post-hoc analysis on this phenomenon but we suspect degree of expertise and the number of burn images could be the culprit”.


Response to Issue No. 3: subjects' body parts for estimation

Accuracy of BurnCase 3D seems to vary depending on the location of patient’s wound. This was not mentioned in our manuscript, which we now included as below. It appears after the 8th sentence of Result section that ends with “and 8 by electric burn.” and also include a new table:

"Out of the 26 subjects, the wounds appeared on different body parts as following: 8 cases in the head; 8 cases in the neck; 16 cases in the trunk (Ant. & Post.); 2 cases in the right buttock; 3 cases in left buttock; 4 cases in the genitalia; 10 cases in the right arm (except hand); 11 cases in the left arm (except hand); 2 cases in the right hand; 3 cases in the left hand; 8 cases in the right leg; 7 cases in the left leg; 4 cases in the right foot; 3 cases in the left foot. Only 3 cases had wounds near the anus or the perineum (Table 2)."

The following sentence has been newly added to Discussion section, after the second paragraph. "One known shortcoming of BurnCase3D is that estimation result can be inaccurate when the wound is located where different body parts overlap, e.g., perineum or the medial side in the buttock. Only 3 out of the 26 patient cases had wounds near the buttock or the perineum in our experiment. In other words, the sample size is too small for claiming that BurnCase 3D has a weakness in estimating burn areas around either areas because 23 of our cases were outside of either regions."


Response to Issue No. 4: Grouping patients with TBSA threshold of 20% and comparing them

In our study, only 5 out of the 26 subjects had TBSA greater than 20% as determined with the Lund-Browder chart. We agree that the estimation results of BurnCase3D results may diverge above the TBSA threshold of 20%, however, our statistician who performed the post-hoc analysis does not agree. We concluded that a larger size of subjects and participants are what we need in our future work, and this was stated in Discussion as following:


"Out of the 26 subjects, 5 had TBSA greater than 20% according to the Lund-Browder chart and the remainder, 21 subjects had TBSA less than 20%. Discussion with our statistician for post-hoc analysis has concluded that we may not know with confidence whether the two groups are different. Our future work will research with more subjects and more participants."

--Stonegaze (discusscontribs) 03:09, 9 January 2019 (UTC)

Editorial comments[edit]

Comments by Thomas Shafee
These comments were submitted on , and refer to this previous version of the article

Thank you for your responses to the peer reviewer comments. A couple of minor points have been raised by the editorial board:

  • The abbreviation TBSA would be best written in full in the title and the first time in the abstract. Ditto PC in the introduction, etc. Consider participants versus participant in the methods section and other such copy editing changes.
  • There are six tables. Are any presentable as figures e.g. the Inter-observer variability tables?
  • The journal uses the style of punctuation-before-references.
  • We recommend adding ORCIDs for the remaining authors (registration for those currently without).
  • Where possible, tables should be made sortable for added functionality.
  • The conclusions should include at least a sentence, and maybe a paragraph, about “Limitations and future directions”.
  • “Participants” is an ambiguous term in the tables. I would suggest switching to “Rater” or “Observer” for the column headings to keep the role distinct from patients (who are also contributing to the data).
Response

Thank you for this insightful comment. We have revised the manuscript reflecting your words.

  • Yes, that's correct. I made changes to the manuscript as you recommended. However, I am embarrassed to say this but is it possible to change the title? My understanding is that is is not so.
  • Can your question be paraphrased as ‘Pick a table that represents all of the 6 tables.’? If it is so, then Table 6 is the representative of all. Please let me know if I am mistaken.
  • Thank you for pointing out this mistake. I corrected the punctuations.
  • We added ORCID for all authors.
  • I noticed that you already sorted the table. Thank you!
  • Reflecting your comment, we revised the manuscript as following:
...when the BurnCase 3D was used.  One limitation of this study was that there were very few patients with TBSA 20% or above and few had burns on perineum, making our post-hoc analysis less robust for evaluating BurnCase 3D.  The meaningful discrepancy in user proficiency for using BurnCase 3D among observers and weak diversity of burn areas were additional factors that could be improved in the future.

The next research would include a larger number of observers with varied proficiencies; more in-depth evaluation method for BurnCase 3D; diverse burn areas as well as specific body parts. Since accuracy of %TBSA estimation is related to prognosis, correlation of %TBSA estimation versus mortality may meaningfully contribute to future work.
  • We are now using 'observer' only and it enhances clarity.

Comments by Thomas Shafee
These comments were submitted on , and refer to this previous version of the article

Thank you for addressing all of these. The title edit is no problem at all - I've now implemented via a "page move".

Comments by Mark Worthen
These comments were submitted on , and refer to this previous version of the article

Copy edit suggestion

CURRENT:

Background: The objective of this study is finding out time requirement and discrepancy in estimation for TBSA burned among burn experts when using the Lund-Browder chart and the BurnCase 3D®.

SUGGESTED REVISION:

Objective: Measure time required to determine total body surface area (TBSA) burned (%TBSA) using the Lund-Browder chart and BurnCase 3D®, and calculate variance between the two methods' %TBSA estimates.

Inter-rater reliability

Since the article promises to compare the *consistency* of the Lund-Browder Chart and the BurnCase 3D®, I want to make sure the appropriate statistics were computed to measure such consistency. (I should acknowledge right away that I'm not a "stats guy", so my apologies in advance my concerns are unwarranted.)

In the Material and methods section, the authors state, "Repeated ANOVA measure was used for evaluating inter-observer variability. Statistical analysis was performed using the SAS® version 9.4 for Windows." And in the Results section, the authors report, " Inter-individual variation was tested with Repeated ANOVA measure and we found no statistical significance (p = 0.31)."

If I understand correctly, reporting "no statistical significance" is not what we need to know. We need to know the inter-rater reliability, computed as an intraclass coefficient (ICC). Here's a quote from Hallgren (2012):

"Significance test results are not typically reported in IRR studies, as it is expected that IRR estimates will typically be greater than 0 for trained coders (Davies & Fleiss, 1982)." (p. 11 of the author's manuscript on PMC)

The ICC should be readily available from the authors' SAS output, although we should have someone well-versed in determining inter-rater reliability check if the correct model, type, unit, and effect were chosen for the analysis.

So that it's clear what I mean, here's my non-statistician guess at what should have been chosen for the analysis:

model: two-way

type: absolute agreement

unit: average-measures

effect: mixed


References

Hallgren KA. Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutor Quant Methods Psychol. 2012 ; 8(1): 23–34. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3402032/

Davies M, Fleiss JL. Measuring agreement for multinomial data. Biometrics. 1982; 38(4):1047–1051.

Response

Thank you for your comment about Material and Methods section. We agree with your comment. Following is our revision in Material and Methods:

Statistical analysis was performed using the SAS® version 9.4 for Windows. In order to compare the results based on Lund-Browder chart and the BurnCase 3D, we performed a paired t-test. In consideration of inter-observer variability, we used repeated ANOVA measure, in which we opted for the default setting. To verify differences among the three groups, we used one-way model, in which we chose average and nested-effects for unit and effect, respectively.