Predictive Relationships Between Concert Band Size and Ratings at Adjudicated Music Performance Assessments

Assessing musical performance is a common practice throughout the field of music education. Directors often use ensemble performances, such as marching band competitions or concert band music performance assessments (MPAs), as a tool to gain feedback and determine if students are reaching the necessary benchmarks (Werpy, 1995). Ratings at these performance assessments can be important to parents, administrators, and the community (Boyle, 1992), and in many situations, director job security and perceptions of their ability are tied to ensemble ratings (Burnsed et al., 1985; Rohrer, 2002). Therefore, the reliability of these assessments is important to ensure that they continue to benefit both students and teachers. Many researchers have studied various aspects of MPAs to investigate whether the rating scales and procedures used accurately measured the quality of ensemble performance (e.g., Burnsed et al., 1985; Stanley et al., 2002). Some researchers (King & Burnsed, 2009; Rickels, 2012) examined the relationship between marching band size and performance ratings and found that larger marching bands earned significantly higher ratings than smaller marching bands. Similarly, larger choirs tended to receive higher ratings than smaller choirs (Killian, 1998, 1999, 2000). In an experimental setting, Silveira and Silvey (2020) attempted to control performance variables while investigating the effects of band size and repertoire difficulty on performance ratings of concert bands and found a three-way interaction effect between band size, repertoire difficulty, and presentation order. Although it has been studied in an experimental setting, no research has been found that examined that relationship in live concert band performances at adjudicated MPAs. Therefore, the purpose of this study was to determine the relationship between ensemble size and ratings earned by middle school and high school concert bands at live adjudicated music performance assessments. Specifically, I examined (a) the profile of ratings assigned to concert band performances at adjudicated music performance assessments, (b) the size of the ensembles that received each rating, (c) the relationship between ensemble size and assigned ratings, and (d) the extent to which band size predicts overall ratings. The extant data for this study consisted of scores earned at district-level music performance assessments (MPAs) by Florida middle school (n = 2,788) and high school (n = 2,782) concert bands between the years 2014 and 2019. Data included concert ratings from a three-member adjudication panel (N = 16,710), a sight reading rating from a single judge (N = 5,570), and overall performance ratings (N = 5,570). I obtained the size of each ensemble, grade level (i.e., middle school or high school), and the ratings earned from a spreadsheet publicly available on the Florida School Music Association’s website (Florida School Music Association, 2020). Given the ordinal nature of division ratings, a Spearman correlation coefficient was conducted to determine the relationships between ensemble size and overall performance ratings of middle school and high school concert bands. To determine whether band size was a predictor of ensemble performance ratings at adjudicated MPAs, I conducted an ordinal logistic regression. One quantitative predictor variable, band size (which ranged from 7 to 128), was used in this analysis. Overall ratings of Superior and Excellent (82.26%, n = 4,582) were assigned to most ensembles at both the middle school and high school levels, whereas the bottom performance rating of Poor was assigned to fewer than one percent of bands (.17%, n = 10). Spearman correlation coefficients indicated that there were moderate negative relationships (rs ranged from -.36 to -.30, p < .001) between ensemble size and performance ratings in all of the comparisons suggesting that as ensemble size increased, ratings decreased (improved). Additionally, an ordinal logistic regression analysis indicated that band size provided a statistically significant prediction of overall MPA ratings (p < .001) with 13% of the total variance in ratings being attributable to the size of the band. The profile of ratings assigned at district-level MPAs suggests that either ensembles performed well or perhaps that generosity error resulted in adjudicators being more lenient when assigning ratings. Judges should continue to combat generosity error and assign honest ratings that accurately reflect the quality of the performances being adjudicated. Given the Spearman correlation and ordinal logistic regression results, there seems to be a predictive relationship between ensemble size and performance ratings, suggesting that smaller ensembles may be somewhat disadvantaged at music performance assessments. However, ensembles of various sizes earned each rating implying that there are factors other than ensemble size that contribute to ratings. Although one could argue that “good band sound is good band sound,” the limited instrumentation of smaller ensembles may make it more difficult for those groups to achieve a typical full band sonority. Smaller bands could also have more difficulty demonstrating stronger dynamic contrasts. Additionally, balance issues could be present in these smaller ensembles due to the limited instrumentation or the maturity of performance skills. On the other hand, smaller ensembles may have benefits such as more opportunities for one-on-one attention with students and more individualized instruction time. Directors of these ensembles should continue to make adjustments and manipulations where they see fit to create a good band sound, balance, and dynamic variation with limited instrumentation and players. Likewise, judges may need to be more forgiving when directors choose to alter parts or scoring to better fit their ensemble. To mitigate the effects of ensemble size on performance ratings, perhaps a portion of adjudicator training sessions could be devoted to discussions of ensemble size, with reminders being given that all bands can have the potential for a high-level of achievement, regardless of size. Evaluating ensembles blindly, where the performing group is hidden visually, may also help diminish the biases that are tied to these visual aspects of performance (Silveira & Silvey, 2020). Given the importance placed on MPAs by parents, administrators, and the community (Boyle, 1992), ongoing research is needed to continue to refine the adjudication process and continue to provide the best possible instruction to students.   References Boyle, J. D. (1992). Program evaluation for secondary school music programs. NASSP Bulletin, 76(544), 63–68. https://doi.org/10.1177/019263659207654413 Burnsed, V., Hinkle, D., & King, S. (1985). Performance evaluation reliability at selected concert festivals. Journal of Band Research, 21(1), 22–29. Florida School Music Association (2020). Literature Performed. Retrieved from https://flmusiced.org/mpaonline/publicreports/literatureperformed.aspx Killian, J. (1998). Characteristics of successful choirs in a contest setting. Texas Music Education Research, 39–43. http://www.tmea.org/assets/pdf/research/Kil1998.pdf Killian, J. (1999). Music selection of successful choirs at UIL and non-UIL contests. Texas Music Education Research, 51–56. http://www.tmea.org/assets/pdf/research/Kil1999.pdf Killian, J. (2000). Effect of music selection on contest ratings: Year three of a continuing study. Texas Music Education Research, 56–61. http://www.tmea.org/assets/pdf/research/Kil2000.pdf King, S. E., & Burnsed, V. (2009). A study of the reliability of adjudicator ratings at the 2005 Virginia band and orchestra directors association state marching band festivals. Journal of Band Research, 45(1), 27–32. Rickels, D. A. (2012). Nonperformance variables as predictors of marching band contest results. Bulletin of the Council for Research in Music Education, 194, 53–72. https://doi.org/10.5406/bulcouresmusedu.194.0053 Rohrer, T. P. (2002). The debate on competition in music in the twentieth century. Update: Applications of Research in Music Education, 21(1), 38–47. https://doi.org/10.1177/87551233020210010501 Silveira, J. M., & Silvey, B. A. (2020). Effects of ensemble size and repertoire difficulty on ratings of concert band performances. Journal of Research in Music Education. Advance online publication. https://doi.org/10.1177/0022429420908280 Stanley, M., Brooker, R., & Gilbert, R. (2002). Examiner perceptions of using criteria in music performance assessment. Research Studies in Music Education, 18(1), 46–56. https://doi.org/10.1177/1321103X020180010601 Werpy, S. F. (1995). Relationships between selected factors of motivation for participation in high school band and general motivation for musical experience (Publication No. 9537530) [Doctoral dissertation, Northwestern University]. ProQuest Dissertations and Theses Global.