Ses kayıtlarından otomatik keman müzik transkripsiyonu

Year 2023, Volume: 14 Issue: 2, 229 - 246, 20.06.2023

Kemal Avcı , Tamer Şevki Acuner

https://doi.org/10.24012/dumf.1246822

Abstract

Bu çalışmada, başlangıç seviyesi keman eğitiminde kullanmak amacıyla keman öğrenenlere çalma performanslarıyla ilgili bir geri dönüt sağlayacak karmaşık spektral fark yöntemi tabanlı bir otomatik müzik transkripsiyon sistemi önerilmiştir. Ayrıca, önerilen müzik transkripsiyon sistemine dayalı ve kemandaki temel etütlerden olan dört boş tel, sol majör arpej ve sol majör dizi notalarını tespit eden Matlab yazılım tabanlı bir kullanıcı arayüzü gerçekleştirilmiştir. Önerilen sistemin performans analizi için iPad tablet tabanlı profesyonel kayıt sistemi kullanarak sekiz katılımcıdan elde edilmiş bir ses kayıt veri seti oluşturulmuştur. Önerilen sistemin keman ses kayıtlarının analizini doğru yapabilmesi için müzik parçasının kendisini oluşturan notalara uygun bölütlenmesi, bunun için de notaların başlangıç zamanının doğru bir şekilde tespit edilmesi gerekmektedir. Piyano ve gitar gibi diğer müzik çalgı seslerine kıyasla, keman sesinin nota başlangıç zamanı tespiti, sahip olduğu zarf karakteristiği nedeniyle daha zordur. Önerilen çalışmada nota başlangıç zamanı tespiti için karmaşık spektral fark yöntemi kullanılmaktadır. Daha sonra, çıkarılan bölüte hızlı Fourier dönüşümü uygulanarak keman sesinin notası ve oktavı belirlenecek şekilde bölütün temel frekansı bulunmaktadır. Ayrıca, geliştirilen arayüz üzerinde süre ve gürlük analizleri de yapılabilmektedir. Kıyaslamalı sonuçlar, önerilen sistemin önemli müzik analiz yazılımları olan MIRtoolbox ve Essentia’daki yöntemlere göre daha başarılı performans sergilediğini göstermektedir.

Keywords

Müzik işaret işleme, Otomatik müzik transkripsiyonu, Keman, Başlangıç zamanı tespiti, Matlab, Kullanıcı arayüzü

References

• [1] Parker B. Güçlü Titreşimler - Müziğin Fiziği. Birinci baskı. Ankara, Türkiye, Tübitak Yayınları, 2015.
• [2] Çuhadar CH. “Kemanda çalma teknikleri”. Çukurova Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 18(1), 121-132, 2009.
• [3] Uçan A. Müzik Eğitimi: Temel Kavramlar-İlkeler-Yaklaşımlar. Ankara, Türkiye, Müzik Ansiklopedisi Yayınları, 1997.
• [4] Öz NB. “İnsanın kültürel gelişiminde müzik eğitiminin önemi”. Uludağ Üniversitesi Eğitim Fakültesi Dergisi, 14(1), 101-106, 2001.
• [5] Çuhadar CH. “Müzik ve beyin”. Çukurova Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 17(2), 67-76, 2008.
• [6] Durahim AO, Setirek AC, Özel BB, Kebapçı H. “Music emotion classification for Turkish songs using lyrics”. Pamukkale Üniversitesi Mühendislik Bilimleri Dergisi, 24(2), 292-301, 2018.
• [7] Hallam S. “The power of music: its impact on the intellectual, social and personal development of children and young people”. International Journal of Music Education, 28(3), 269-289, 2010.
• [8] Topalak Ş. “Güzel sanatlar lisesi çalgı eğitimi/öğretiminde karşılaşılan sorunların incelenmesi”. Sanat Eğitimi Dergisi, 1(2), 114-129, 2013.
• [9] Qionggang R. “Application and research on digital music technology in music teaching”. Computer-Aided Industrial Design & Conceptual Design, Wenzhou, China, 26-29 November 2009.
• [10] Liang YC, Shiue YM, Huang YM, Liu CG. “Development of a digital game-based learning system in music learning”. Advanced Materials for Science and Engineering, Tainan, Taiwan, 12-13 November 2016.
• [11] Pati KA, Gururani S, Lerch A. “Assessment of student music performances using deep neural networks”. Applied Sciences, 1-8(4), 1-18, 2018.
• [12] Klapuri AP. Signal Processing Methods for the Automatic Transcription of Music. PhD Thesis, Tampere University of Technology, Tampere, Finland, 2004.
• [13] Muller M, Ellis DP, Klapuri A, Richard G. “Signal processing for music analysis”. IEEE Journal of Selected Topics in Signal Processing, 5(6), 1088-1110, 2011.
• [14] Muller M. Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications. 1st ed. Switzerland, Springer International Publishing, 2015.
• [15] Chis LG, Marcu M, Dragan F. “Software tool for audio signal analysis and automatic music transcription”. IEEE 12th International Symposium on Applied Computational Intelligence and Informatics (SACI), Timisoara, Romania, 17-19 May 2018.
• [16] Benetos E, Dixon S, Duan Z, Ewert S. “Automatic music transkriptin: an overview”. IEEE Signal Processing Magazine, 36(1), 20–30, 2019.
• [17] Wu YT, Luo YJ, Chen TP, Wei IC, Hsu JY, Chuang YC, Su L. “Omnizart: a general toolbox for automatic music transcription”. arXiv, 2106.00497 [cs.SD], 2021
• [18] Argenti F, Nesi P, Pantaleo G. “Automatic transcription of polyphonic music based on the constant-q bispectral analysis”. IEEE Transactions on Audio, Speech, and Language Processing, 19(6), 1610-1630, 2011.
• [19] Benetos E, Dixon S, Giannoulis D, Kirchhoff H, Klapuri A. “Automatic music transcription: breaking the glass ceiling”. International Society for Music Information Retrieval Conference, Porto, Portugal, 8-12 October 2012.
• [20] Avci K, Acuner TŞ. “Keman Kayıtlarından Boş Tel Notalarının Otomatik Transkripsiyonu”, 25. IEEE Sinyal İşleme ve İletişim Uygulamaları Kurultayı (SIU-2017), Antalya, Türkiye, 15 – 18 Mayıs 2017.
• [21] Bello JP, Daudet L, Abdallah S, Duxbury C, Davies M, Sandler MB. “A tutorial on onset detection in music signals”. IEEE Transactions on Speech and Audio Processing, 13(5), 1035-1047, 2005.
• [22] Moorer JA. “On the transcription of musical sound by computer”. Computer Music Journal, 1(4), 32-38, 1977.
• [23] Piszczalski M, Galler BA. “Automatic music transcription”. Computer Music Journal, 1(4), 24-31, 1977.
• [24] Klapuri AP. “Automatic music transcription as we know it today”. Journal of New Music Research, 33(3), 269-282, 2004.
• [25] Benetos E, Dixon S, Giannoulis D, Kirchhoff H, Klapuri A. “Automatic music transcription: challenges and future directions”. Journal of Intelligent Information Systems, 41(3), 407-434, 2013.
• [26] Tavares TF, Barbedo JGA, Attux R, Lopes A. “Survey on automatic transcription of music: Historical overview of techniques”. Journal of the Brazilian Computer Society, 19(4), 589-604, 2013.
• [27] Gowrishankar B, Bhajantri NU. “An exhaustive review of automatic music transcription techniques: survey of music transcription techniques”. Signal Processing, Communication, Power and Embedded System, Odisha, India, 3-5 October 2016.
• [28] Yazawa K, Itoyama K, Okuno HG. “Automatic transcription of guitar tablature from audio signals in accordance with player's proficiency”. IEEE International Conference on Acoustics, Speech and Signal Processing, Florence, Italy, 4-9 May 2014.
• [29] Abesser J, Schuller G. “Instrument-centered music transcription of solo bass guitar recordings”. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(9), 1741-1750, 2017.
• [30] Kehling C, Abeßer J, Dittmar C, Schuller G. “Automatic Tablature transcription of electric guitar recordings by estimation of score and instrument-related parameters”. Digital Audio Effects Workshop (DAFx), Erlangen, Germany, 1-5 September 2014.
• [31] Kirkpatrick T, Andreas J, Klein D. “Unsupervised transcription of piano music”. Advances in Neural Information Processing Systems (NIPS 2014), Montreal, Canada, 8-13 December 2014.
• [32] Akbari M, Cheng H. “Real-time piano music transcription based on computer vision”. IEEE Transactions on Multimedia, 17(12), 2113-2121, 2015.
• [33] Wan Y, Wang X, Zhou R, Yan Y. “Automatic piano music transcription using audio-visual features”. Chinese Journal of Electronics, 24(3), 596-603, 2015.
• [34] Gillet O, Richard G. “Transcription and separation of drum signals from polyphonic music”. IEEE Transactions on Audio, Speech, and Language Processing, 16(3), 529-540, 2008.
• [35] Wu CW, Dittmar C, Southall C, Vogl R, Widmer G, Hockman J, Müller M, Lerch A. "A review of automatic drum transcription". IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26(9), 1457-1483, 2018.
• [36] Wei IC, Wu CW, Su L. "Improving automatic drum transcription using large-scale audio-to-midi aligned data". IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, Canada, 6-11 June 2021.
• [37] Souza VM, Batista GE, Souza-Filho NE. “Automatic classification of drum sounds with indefinite pitch”. International Joint Conference on Neural Networks, Killarney, Ireland, 12-17 July 2015.
• [38] Marolt M. “Automatic transcription of bell chiming recordings”. IEEE Transactions on Audio, Speech, and Language Processing, 20(3), 844-853, 2012.
• [39] Skoki A, Ljubic S, Lerga J, Atajduhar I. “Automatic music transcription for traditional woodwind instruments sopele”. Pattern Recognition Letters, 128(2019), 340-347, 2019
• [40] Kroher N, Gomez E. “Automatic transcription of flamenco singing from polyphonic music recordings”. IEEE/ACM Transactions on Audio, Speech and Language Processing, 24(5), 901-913, 2016.
• [41] Dhara P, Rengaswamy P, Rao KS. “Designing automatic note transcription system for hindustani classical music”. International Conference on Advances in Computing, Communications and Informatics, Jaipur, India, 21-24 September 2016.
• [42] Muto Y, Tanaka T. “Transcription system for music by two instruments”. 6th Signal Processing Conference, Beijing, China, 26-30 August 2002.
• [43] Krishnaswamy A, Smith JO. “Inferring control inputs to an acoustic violin from audio spectra”. International Conference on Multimedia and Expo, Baltimore, USA, 6-9 July 2003.
• [44] Charles JA, Fitzgerald D, Coyle E. “Towards a computer assisted violin teaching aid”. International Symposium on Psychology and Music Education, Padua, Italy, 29 November 2004.
• [45] Yin J, Dhanik A, Hsu D, Wang Y. “The creation of a music-driven digital violinist”. 12th ACM International Conference on Multimedia, New York, USA, 10-16 October 2004.
• [46] Yin J, Wang Y, Hsu D. “Digital violin tutor: an integrated system for beginning violin learners”. 13th ACM International Conference on Multimedia, Hilton, Singapore, 6-11 November 2005.
• [47] Vogel BK, Jordan MI, Wessel D. “Multi-instrument musical transcription using a dynamic graphical model”. IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, USA, 19-23 March 2005.
• [48] Boo WJJ, Wang Y, Loscos A. “A violin music transcriber for personalized learning”. IEEE International Conference on Multimedia and Expo, Toronto, Canada, 9-12 July 2006.
• [49] Loscos A, Wang Y, Boo WJJ. “Low level descriptors for automatic violin transcription”. International Society for Music Information Retrieval Conference, Victoria, Canada, 8-12 October 2006.
• [50] Charles JA, Fitzgerald D, Coyle E. “Violin timbre space features”. IET Irish Signals and Systems Conference, Dublin, Ireland, 28-30 June 2006.
• [51] Wang Y, Zhang B, Schleusing O. “Educational violin transcription by fusing multimedia streams”. International Workshop on Educational Multimedia and Multimedia Education, Augsburg, Germany, 28 September 2007.
• [52] Thornburg H, Leistikow RJ, Berger J. “Melody extraction and musical onset detection via probabilistic models of framewise STFT peak data”. IEEE Transactions on Audio, Speech, and Language Processing, 15(4), 1257-1272, 2007.
• [53] Zhang B, Zhu J, Wang Y, Leow WK. “Visual analysis of fingering for pedagogical violin transcription”. 15th ACM International Conference on Multimedia, Augsburg, Germany, 23-28 September 2007.
• [54] Lu H, Zhang B, Wang Y, Leow WK. “iDVT: an interactive digital violin tutoring system based on audio-visual fusion”. 16th ACM International Conference on Multimedia, Vancouver, Canada, 27-31 October 2008.
• [55] Charles JA, Fitzgerald D, Coyle E. “Violin sound quality detection”. IET Irish Signals and Systems Conference, Galway, Ireland, 18-19 June 2008.
• [56] Maezawa A, Itoyama K, Takahashi T, Ogata T, Okuno HG. “Bowed string sequence estimation of a violin based on adaptive audio signal classification and context-dependent error correction”. International Symposium on Multimedia, San Diego, USA, 14-16 December 2009.
• [57] Barbancho I, de la Bandera C, Barbancho AM, Tardon LJ. “Transcription and expressiveness detection system for violin music”. IEEE International Conference on Acoustics, Speech and Signal Processing, Taipei, Taiwan, 19 April 2009.
• [58] Maezawa A, Itoyama K, Komatani K, Ogata T, Okuno HG. “Automated violin fingering transcription through analysis of an audio recording”. Computer Music Journal, 36(3), 57-72, 2012.
• [59] Huang HH. “Research and development of an automatic violin player”. 7th World Congress on Intelligent Control and Automation, Chongqing, China, 25-27 June 2008.
• [60] Huang HH, Li WH, Chen YJ, Wen CC. “Automatic violin player”. 10th World Congress on Intelligent Control and Automation, Beijing, China, 6-8 July 2012.
• [61] Carrillo AP, Bonada J, Maestre E, Guaus E, Blaauw M. “Performance control driven violin timbre model based on neural networks”. IEEE Transactions on Audio, Speech, and Language Processing, 20(3), 1007-1021, 2012.
• [62] Wang JH, Wang SA, Chen WC, Chang KN, Chen HY. “Real-time pitch training system for violin learners”. IEEE International Conference on Multimedia and Expo Workshops, Melbourne, Australia, 9-13 July 2012.
• [63] Lin YJ, Wang TM, Chen TC, Chen YL, Chang WC, Su AW. “Musical note analysis of solo violin recordings using recursive regularization”. EURASIP Journal on Audio, Speech, and Music Processing, 25(2014), 1-13, 2014.
• [64] Pardue LS, Harte C, McPherson AP. “A low-cost real-time tracking system for violin”. Journal of New Music Research, 44(4), 305-323, 2015.
• [65] Jo W, Park H, Lee B, Kim D. “A study on improving sound quality of violin playing robot”. 6th International Conference on Automation, Robotics and Applications, Queenstown, New Zealand, 17-19 February 2015.
• [66] Liang CY, Su L, Yang YH, Lin HM. “Musical offset detection of pitched instruments: the case of violin”. International Society for Music Information Retrieval Conference, Malaga, Spain, 26-30 October 2015.
• [67] Jo W, Lee B, Kim D. “Development of auditory feedback system for violin playing robot”. International Journal of Precision Engineering and Manufacturing, 17(6), 717-724, 2016.
• [68] Maruyama T, Uemura T. “Development of a violin playing robot and its sound volume and pitch controls”. 56th Annual Conference of the Society of Instrument and Control Engineers of Japan, Kanazawa, Japan, 19-22 September 2017.
• [69] Yura J, Oyun-Erdene M, Byambasuren BE, Kim D. Modeling of Violin Playing Robot Arm with Matlab/Simulink. Editors: Kim JH, Karray F, Jo J, Sincak P, Myung H. Robot Intelligence Technology and Applications 4 of Advances in Intelligent Systems and Computing, 249-261, Switzerland, Springer-Cham, 2017.
• [70] Li B, Dinesh K, Sharma G, Duan Z. “Video-based vibrato detection and analysis for polyphonic string music”. International Society for Music Information Retrieval Conference, Suzhou, China, 23-27 October 2017.
• [71] Pardue LS, McPherson A. “Real-time aural and visual feedback for improving violin intonation”. Frontiers in Psychology, 10(627), 1-19, 2019.
• [72] Ortega FJM, Giraldo SI, Perez A, Ramirez R. “Phrase-level modeling of expression in violin performances”. Frontiers in Psychology, 1-11, 10(776), 2019.
• [73] Özmenteş S. “Müzik eğitiminin boyutları ve çalgı eğitimi”. İnönü Üniversitesi Eğitim Fakültesi Dergisi, 6(9), 89-98, 2005.
• [74] Lartillot O, Toiviainen P. “A Matlab toolbox for musical feature extraction from audio”. Digital Audio Effects Workshop (DAFx), Bordeaux, France, 10-14 September 2007.
• [75] Bogdanov D, Wack N, Gómez E, Gulati S, Herrera P, Mayor O, Roma G, Salamon J, Zapata J, Serra X. "Essentia: an open-source library for sound and music analysis". 21st ACM International Conference on Multimedia, Barcelona, Spain, 21-25 October 2013.
• [76] Duxbury C, Bello JP, Davies M, Sandler M. “A combined phase and amplitude based approach to onset detection for audio segmentation”. Workshop on Image Analysis for Multimedia Interactive Services, London, UK, 9-11 April 2003.
• [77] Music Technology Group (MTG). “Essentia”. https://github.com/MTG/essentia/blob/master/src/examples/ standard_pitchdemo.cpp (22.11.2017).
• [78] Brossier PM. Automatic Annotation of Musical Audio for Interactive Applications. PhD Thesis, Queen Marry University of London, London, UK, 2007.
• [79] Salamon J, Gomez E. “Melody extraction from polyphonic music signals using pitch contour characteristics”. IEEE Transactions on Audio, Speech, and Language Processing, 20(6), 1759–1770, 2012.
• [80] Klapuri AP. “Multiple fundamental frequency estimation by summing harmonic amplitudes”. International Society for Music Information Retrieval Conference, Victoria, Canada, 8-12 October 2006.