Innovations in CM using Keyboards

cmlover · Post by **cmlover** » 15 Apr 2006, 22:34

Happy new year everybody!
This is an exciting new topic for everybody who is interested in novel approaches to CM using modern computer wizardry. These new frontiers are as exciting as the possibilities are endless; your imagination is the only limit. In this thread we shall explore the use of keyboards in CM and the adaptation of computers to morph and transform traditional CM into exciting 'new' forms of music. I am neither a musician nor am I a computer expert. I am just an avid admirer of CM but at the same time an 'iconoclast' who wants to break the traditional barriers of CM by promoting discussions and sharing of knowledge in the 'new' areas. Suffice it to say that Dikshitar opened a new chapter in CM with the introduction of the Western Violin in CM which is now an integral part of our concert platform. In recent times Srinivas with the Mandolin and Prasanna with the Electric Guitar have broken new frontiers in CM. These are traditional CM experts who did not fight shy of 'novel' presentations of CM. I/We request the indulgence of the orthodoxy in case we outstep the limits of the traditional bounds. Plese feel free to crticize; but constructively! vandanam, Thanks!

cmlover · Post by **cmlover** » 15 Apr 2006, 22:40

Sahana and Anandabhairavi from Mohan

Right Click and select "Save as"
http://freepgs.com/carnatic/midi/sahana.mid
http://freepgs.com/carnatic/midi/anandabhairavi.mid

As a starter please listen to these MID music graciously posted by mohan. I will post my analysis soon.

cmlover · Post by **cmlover** » 16 Apr 2006, 02:14

Ok! I am sure you listened to both the MID songs. let us take the sahana first. (Note you could play the MID files in your Windows Media Player or Real Audio or several other players!) After hearing it if you said 'Terrible' I certainly would agree with you! This has no reflection on Mohan's ability to play! Then why? Because the computer chooses 'Acoustic Guitar' as the instrument (Yes the same which Prasanna uses!). This instrument (in the computer) is optimised for Western Music and hence the 'horrible' plucking sound is accentuated. Also though mohan uses the gamakams very effectively (the kampitham on the gaandharam and the oscillation upto madhyamam etc.,) he uses portmento (same as glissando which is smooth glide from one note to another) the better strategy is 'pitchbend' (continuously changing the frequency of the pitch) wherein the transitions will be smoother! The 'attack phase' of the Acoustic Guitar (AG) is abrupt which has to be mollified. Now I chose the flute(Western) as the instrument to play the same MID file and here it is:
http://rapidshare.de/files/18079672/sah ... e.mid.html

Isn't it much better? In fact you can choose over a hundred instruments (all Western) to play the MID file. Here is violin
http://rapidshare.de/files/18079743/sah ... n.mid.html

It is OK but the 'tinniness' is characteristic of the Western Instrument. They can be adjusted to a limited extent in a MIDI editor. We need CM trained Computer experts to design typical CM instrument banks! Here is same music on the Sitar accompanied by Flute and Violin.
http://rapidshare.de/files/18094505/sah ... r.mid.html

Not so good of course. But if you edit the MID files to 'mollify' it will sound much better. The size of the MID file is 3KB (No! it is not MB!). How can we play 1.5 minutes of music in such a small file? In fact the MID file does not contain any music like your Wav or MP3 files. It is instructions to the computer to play the notes in an english like language which is very efficient. The MIDI language is easy to learn. We need lots of MIDI programmers for CM who can take our glorious CM into the hightech era of the 21st Century!

cmlover · Post by **cmlover** » 16 Apr 2006, 02:28

cmlover, what is pitchbend ?

As I have explained above 'pichbend' is the continuous variation of the frequency of a note. In fact it makes the digital music into anlog! Most keyboarda have a pitchbend wheel which permit this manipulation. You must have seen a good example of it by Michael Fox in the movie 'Back to the Future' where the electric guitar just screams

Our CM is analog,unlike the Western Music which is digital! That is the reason we cannot use the Piano to play CM (also the keyboard without the pitchbend!)

vasanthakokilam · Post by **vasanthakokilam** » 16 Apr 2006, 03:07

Thanks very much Mohan.

>(Note you could play the MID files in your Windows Media Player or Real Audio or several other players!)

Well, that is what I tried. Quicktime picked it up first, then I tried WMP.

I do not know how to turn on portamento. Without that it does not sound anywhere close to CM ( as Mohan has told us ahead of time ). Sahana was not recognizable by me as such but Anandabhairavi fared much better.

CML, your converting it to different instruments already shows one of the powers of MIDI. I am sure there are many. Including getting a quick notation down and for other educational purposes. This also paves the way for some internet music collaboration ( a different thread probably ) among us.

Playing it without the portamento demonstrated so much about what carnatic music is all about which we discussed in the 'what is a raga' thread.

Now I am on a hunt for a MIDI player with portamento!!

mohan · Post by **mohan** » 16 Apr 2006, 04:34

CML suggested that Midi 2 has portamento? Can you explain how to turn this on?

If I connect my computer back to my synthesizer and play the midi file (on the compuer) but set it up so that the midi device is actually the synthesizer, then I can choose instruments on my synth to play back the sound. If I choose a sound that is set up with portamento then that effect also is played back from the midi file.

Hence, the midi file does seem to have this information embedded in it. CML can you suggest how to enable the portamento on a sound using a computer player. Probably we need something like Cakewalk Pro?

I think if you are able to activate this effect, you will see that portamento is a viable alternative to pitch bend.

Another question: can you change instruments within Windows Media Player?

cmlover · Post by **cmlover** » 16 Apr 2006, 10:53

mohan

Portamento belongs to the older MIDI 1.0, but the MIDI 2.0 uses pitchbend which is versatile. Portamento essentially implements the 'jaru' in CM. But pitchbend implements in addition the frequency change within the notes (eg., varali madhyamam etc.,). The computer program automatically detects whatever you are using in your keyboard and codes it in the MID file. The stccato that you are getting in your MID file is due to the inefficiency of the portamento. If you use pitchbend the transitions will be very smooth. In other words you will be playing keyboard almost like the veeNa using the pitchbend wheel! If somebody can implement the veeNa into the 'instrument bank' of the MIDI one can easily play a perfect veeNa with it!

You cannot change the instruments in the Windows Media plyer or Real audio since they are essentially Wav players. If you use a dedicated MID player (cakewalk, cubase etc.,) you will be able to choose the instruments freely from the dedicated sound bank.

To get the Western notation as the MID plays use this free MIDI player.
http://www.noteworthysoftware.com/player/

vasanthakokilam · Post by **vasanthakokilam** » 16 Apr 2006, 12:49

CML, how do I turn on portamento in WMP or RP? If that is not possible, have you found a MIDI players that can do portamento.

cmlover · Post by **cmlover** » 16 Apr 2006, 23:55

vk
Portamento is recognized by the MIDI language interpreter. It is always on. If you are playing back through your 'keyboard' then you have to enable it on the keyboard! Computers simply obey the commands of the MIDI language. Again Portmento belongs to MIDI 1.0 which is superceded by MIDI 2.0 which has the pitchbend. But the guy playing has to use the pitchbend to have it recorded. For example Satya has used pitchbend in his Todi aalaapana. But he has not given us a MID file! I am experimenting with his file and will post later. I hope mohan gets to use pitchbend! Try the Noteworthy player (it is free) which gives you the western notations which wll give a feel for what is happening!

vasanthakokilam · Post by **vasanthakokilam** » 17 Apr 2006, 01:11

CML, thanks. The Noteworthyplayer is really cool. I can not really read the western notation. ( though my 9 year old niece has promised to teach me, she says it is a piece of cake ). But the player shows where it is in the score and I can make some sense out of the up and down trend. It is a graphical representation anyway..So far so good.

The reason I asked about 'turning on' Portamento is, I do not hear any such Jaru gamaka effect at all, either in WMP or in Noteworthy Player. Mohan's concert mp3 file had all that effect. Without Portamento, Sahana.mid does not have any Sahana at all. 'anandabhairavi.mid' fared better in bringing out some anandabhairavi. And Mohan says that when he played the same file through the keyboard, it faithfully reproduced the Portamento effects. So what is going on?

vasanthakokilam · Post by **vasanthakokilam** » 17 Apr 2006, 01:21

CML, I have a feeling you will eat this up.

http://www.intelliscore.net/

The Intelliscore converts other formats like mp3 to MIDI. The free version only converts the first 15 seconds.

So, by combining this with Noteworthy player we can get an mp3 CM file notated in western notation!! At least it can help in getting started for those like me who can not translate music to notes by hearing. And then one can apply the gamakas of the raga in question.

All, if you get a chance to try it, please post your experiences. Also please post any other tools out there that converts to MIDI.

cmlover · Post by **cmlover** » 17 Apr 2006, 02:23

vk

The dream of musicians ever since the computer was to invent a 'machine' that will automatically midify music. None axists at present. The intelliscore is just useless. The one that now reigns the market is Digital Ear (it is expensive and I have it!). It does not work well with CM because of our gamakas. Works reasonably well with western (instrumental) music. The solution is to use pitchbend which is recognized. Here is the tentative experimental results. (Note that the music has to be monophonic for computer recognition since polyphony will fool the recognition engine!). Here is Satyas Todi aalapana posted
http://rapidshare.de/files/17428262/Tho ... a.mp3.html

Here is what Digital ear produced (using Violin (western) as the instrument)
http://rapidshare.de/files/18178843/TodiKeyDE1.mid.html

The hiccoughs you hear are the artefacts by the computer. They can be eliminated inside a MIDI editor (note-by-note). I have left them as it is. I think it is reasonable (if your tolerance is as high as mine)

.

What do you say mohan?

vasanthakokilam · Post by **vasanthakokilam** » 17 Apr 2006, 02:56

CML:

Digital Ear did a much better job, especially in the beginning and towards the end. I agree that intelliscore is useless. It did marginally OK on monophonic violin without gamakam but otherwise it is quite awful.

( I work with speech recognition and I understand the difficulties with music recognition... each with its own peculiar challenges )

Did you notice that the noteworthy player could not really notate this MIDI file properly?

cmlover · Post by **cmlover** » 17 Apr 2006, 04:18

Sorry sunayana! Commercial ones not permitted here!

vk
I am glad you noticed that! There are no notations for gamkams in Western Music except the 'frills' and legatos. Digital Ear produces most of the gamakams by taking a note (eg., G#) and pitch bending it and Noteworthy is unable to show the notes though plays them faithfully!

We need our CM computer wizards to write programs to recognize CM gamakas and notate them! We still depend on SSP, a century old

There is a crying need!

meena · Post by **meena** » 17 Apr 2006, 04:19

cml PLEASE check mod thread.

vasanthakokilam · Post by **vasanthakokilam** » 17 Apr 2006, 05:29

CML,

Can you feed the file NAYANA1.wav to Digital Ear and see what it produces? This is the file you made for the Kanthaswara krithi by DRS.

cmlover · Post by **cmlover** » 17 Apr 2006, 07:20

here it is
http://rapidshare.de/files/18195906/NAYANA011.mid.html
Note there is no pitchbend used since the original file was created by painting the pianokeys (note by note) on cakewalk! Digital ear is trying to approximate using 1/16th notes (see noteworthy player) and hence the jitteriness. Also the wave file was noisy!
The instrument used is piccolo!

vasanthakokilam · Post by **vasanthakokilam** » 17 Apr 2006, 08:07

Thanks CML. Interesting experiment. I notice the same thing you mentioned. In addition, atleast in two places, there was apaswaram when noteworthy played it.

Did you compare how much the output notation from noteworthy differed from the input you gave to cakewalk ( adjusting for the 1/16th split notes )? If it is ardous work for the whole song, you can take a sample and see how close it came.

Automatic Music transcription is a tough cookie...

mohan · Post by **mohan** » 17 Apr 2006, 09:56

Since the midi experiment didn't work very well with portamento. I have saved an mp3 file of a short Sahana alapana. You can hear the portamento throughout the piece which is available at:

http://www.carnaticcorner.com/sounds/mohan/

CML - you can try to pass this through a program and convert to midi. Sruthi is 3.

vasanthakokilam · Post by **vasanthakokilam** » 17 Apr 2006, 10:16

Mohan, what a difference the mp3 makes. The Sahana, latent and hiding in those swaras blooms so much once the portamento kicks in.

Since you said there is evidence that the sahana.mid file has portamento encodings in it, I am questioning if the MIDI interpreters in the players I used really understand Portamento. I read in one place that portamento is often not implemented by MIDI devices. May be that is true of these software players as well. In fact, one MIDI player blurb I read said that they convert Portamento to Pitch Bend since many devices do not support it.

cmlover · Post by **cmlover** » 17 Apr 2006, 21:44

mohan/vk

here is an analysis of the sahana. First the raw conversion provides
http://rapidshare.de/files/18241637/sahana1.mid.html
This simply captures the notes! Since sahana is not a scalar raga this is terrible but the notes of sahana are noticeabble from time to time.

Now the one with pitchbend
http://rapidshare.de/files/18241695/sahana.mid.html
much better! The kaarvai is from the way you have played. The important gamakams have been captured and sahana is quite visible.

The noteworthyplayer simply displays the notes and no gamkams are visible.
Goes to prove how gamkam is the life and soul of CM. Shall we say:
'CM without gamakams is a flower without fragrance'!
(note that the portamenteau is interpreted in MIDI 2.0 using the pitchbend!)

vasanthakokilam · Post by **vasanthakokilam** » 18 Apr 2006, 00:09

CML, thanks again.

>flower without fragrance...

yes... like all the flowers in the colder regions of NA. My biggest disappointment with this part of the world. Even home grown Jasmine does not have that much fragrance... I am digressing....

Can you clarify what you did differently from the first one and the second one. Is that just a setting in Digital Ear to capture and encode pitch bend ( on the second one )?

I think we will all agree that the level of reco and repro in both cases ( impressive as it is ) is basically at the standard of a beginning student on Violin..

Interestingly, if the original was made with pitchbend rather than portamento, Digital ear seems to do a better job ( based on the two samples we have seen so far ).

cmlover · Post by **cmlover** » 18 Apr 2006, 07:52

vk/mohan/Sivarman

Here is a realtime experiment. Refer to the Raga thread to locate Sivarman's rendering of Varnarupini. Here is his viruttam part
http://rapidshare.de/files/18281866/Siv ... m.mp3.html
I had extracted it with some filtering since his recording is overmodulated. I used digital ear to conver his rendering into MID file (violin as the instrument). Here it is
http://rapidshare.de/files/18281896/Vir ... n.mid.html

It is very reasonable. Of course it is more bass since his voice is about one kaTTai. The pitchbend has reasonably captured his gamakams.

here is Sivaraman accompanied by the violin
http://rapidshare.de/files/18281948/Siv ... d.mp3.html
Since the violin is at the same shruti of Sivarman and playing concurrently you can hear the resonance!

Of course this is totally experimental. If our computer wizards design a good engine we may succeed in eliminating the accompanists and replace them with a computer in real time! (Just kidding)

vasanthakokilam · Post by **vasanthakokilam** » 18 Apr 2006, 08:33

:Applause: CML....

That is great. Digital Ear did a great job on Shri. Sivaraman. I was going to ask you in disbelief if you are just messing with us

It has a humming feel to it or may be a bee whizzing around type of sound but with gamakam...

That is a great idea to have the midi converted file accompany Shri. Sivaraman. May I ask you to do one additional thing. Can you offset the violin slightly ( later in time ) so it gives the illusion of accompaniment, like a violin normally does for a viruttam? Just to see how that sounds. You can move around the offset a bit until it sounds reasonable.

(Kudos to Sivaraman for singing excellently and contributing to this effort )

How does the notation look? Any good? ( I can not tell if noteworthy notation is correct or not, even outside of gamakams. I will ask my niece one of these days ).

I think we have enough data here for a paper in a music conference

cmlover · Post by **cmlover** » 18 Apr 2006, 09:47

Vk

here it is with about 0.5 secs offset
http://rapidshare.de/files/18287483/siv ... 2.mp3.html

They don't go well together because of dissonance (nonshruti alignment). In real accompaniment the violinist will adjust to the vocalist in notes and the pauses; but the computer is dumb! We need to build an atrificial intelligence machine to implement that! Also the Western violin is terrible in comparison with our CM one!

There are lots of exciting possibilities if our bright youngsters will apply themselves to the technology! First they should learn to like and understand CM

vasanthakokilam · Post by **vasanthakokilam** » 18 Apr 2006, 11:05

Not too bad CML..I had to get used to the carbon copy accompaniment but after half way through it started to feel a bit more like a concert setting. Did you play around with it and picked 0.5? How did other offsets sound?

The AI you talk about can be something "simple" as deliberately introduced random but small variations in the created MIDI file. And the challenge is to do it in such a way that it still stays in the same raga. Easier said than done, I suppose.

I am still in awe of digital ear's gamaka preserved midi file generation. If we can get a good notation out of it, that will be an useful educational tool for CM.

vasanthakokilam · Post by **vasanthakokilam** » 18 Apr 2006, 11:41

Just for grins...

For pitch bend purposes, MIDI divides each semitone into 16384 equal steps. That is 2 power 14. And for the whole octave it is 196608 intervals. So each interval is 196608th root of 2 or a ratio of 283608/283607. Thus it can capture and encode/notate any gamakam. The limitation is only in the resolution of the recognizer. And raga specific sharpening or flattening ( Begada, Gowlipantu,...,... ) is not a big problem obviously.

The next exercise is to provide a user friendly way of depicting the pitch-bend values. It already has the numbers that connects the two notes. May be an accurate curvy line can be drawn between the two notes to show the transition and it is upto the performer to translate that to their instrument. And for the oscillatory gamaka, a mouse click on that note can show the actual graph of the oscillation. This is all for practice, not during performance.

The conclusion I am basically drawing is, we do not require any major leap in technology to automatically convert and notate monophonic CM including all the gamakas. The notation step is to go from the rich midi file to a good and usable user interface presentation. That is, it is a matter of a good user interface and presentation design and then it is a matter of programming. The presentation of the notation is not limited to paper, so we can put the current day technology to good use. Only our imaginatin and creativity is the limit.

(At the risk of stating the obvious: Since this is a technical section, we talk about all this technology but it is not a replacement for the art, manodharma, bhavam etc. These technologies are just some educational and illustrative tools to learn and may be to demystify some aspects of CM)

cmlover · Post by **cmlover** » 18 Apr 2006, 22:15

vk

The offset does not matter. Two identical waveforms when mixed with an offset will produce a 'beat'. The beat frequency is a nuisance! In real performance the vocalist and accompanist adjust to each other. Also the accompanist will feed the right place for the vocalist to take-off so that there is unison. If sivaraman were listening to the computer as a 'karaoke' he would do a great job!

I agree we do not need any new technology but the existing ones have to be adopted and adapted!

Now listen to Sivarman feminized (P Susheela

)
http://rapidshare.de/files/18331451/Siv ... d.mp3.html

This is a simple frequency shift (by 4 kaTTai).

Now I must apologize to him for taking liberty with his voice for experimentation. I do love his original voice , of course!

Vocalist · Post by **Vocalist** » 18 Apr 2006, 22:25

It was a nice thought, cml, to coordinate the accompaniment - but i still personally prefer Sivaraman's voice just as is.

I too initially thought the idea of offsetting the violin would be good, like vk. We both forgot that there would be a harsh clash between the notes though, as displayed later.

I am not impressed with the feminising CML!!

cmlover · Post by **cmlover** » 18 Apr 2006, 22:35

Vocalist

it is only for demonstration purposes! It has to be applied judiciously. Have you heard S Balachandar doing a duet with himself (in both high and low octaves simultaneously!). He was a wizard!

We have to explore new frontiers in CM using modern technology and expand the horizons!

Vocalist · Post by **Vocalist** » 18 Apr 2006, 22:38

True. Demo purposes. I'm expressing my perspective as a listener (as almost always).

vasanthakokilam · Post by **vasanthakokilam** » 19 Apr 2006, 02:43

Yes, demo, experimentation, learning etc.

In your research if you come across a good program that converts MIDI file to staff ( or other ) notation which takes into account pitch bend values, please post.

vasanthakokilam · Post by **vasanthakokilam** » 19 Apr 2006, 10:47

I went back to Arvindh Krishnaswamy's paper: http://ccrma.stanford.edu/~arvindh/cmt/ismir04.pdf

There he mentions:

"....We believe that the various entities we have published thus
far are a necessary and sufficient set to be able to represent
or synthesize any phrase in Carnatic music.........For many applications, such as transcription and classification this is exactly what is required."

CML's experiment with Sivaraman's viruttam clearly shows that Digital Ear can convert CM ( for cases without accompaniments ) to MIDI pretty well. If we want to rope in Arvindh's concepts here, three things will complete the picture.

a) Come up with pitchbend representations for these various melodic entities.

b) Once we have that, then the midi file can be converted to a notation that involves these melodic entities.

c) Third, going the other way, take a notation written using these melodic entities and convert to the equivalent MIDI using the pitch bend equivalents. This is probably the easiest of the three.

arunk · Post by **arunk** » 19 Apr 2006, 18:34

vk,

I have actually dabbled with this a bit i.e to be able to (grossly approximate i must emphasize) represent a gamakam as a pitch curve and then use pitch bend to "synthesize" it. But unfortunately

like most of my "experiments" it is in a half-baked state as i moved on to other "half-baked" ideas

I can see how much it will take to resurrect this particular one and put on the web somewhere. It is a java app and so i may be able to convert it to an applet.

But dont hold your breath in anticipation!!! It is a pretty abstract experiment - it lets you define arbitrary curves and then you can synthesize them to find out "how they sound" and thus perhaps be able to define gamakas. There is more to gamakas than just pitch variance and this one doesnt get into any of that. Anyway, the idea was this would help you define building blocks and hence is sort of step 1 out of "n" steps needed for what you guys have been discussing.

Arun

cmlover · Post by **cmlover** » 19 Apr 2006, 20:43

Arun 'was' the brightest star on the CM 'technology' firmament! For whatever reason he faded I don't know. If there ever is a think tank under his guidance we will have a 'superb' automated CM engine. I can only dream about it!
(Though Arvindh is very bright and highly knowledgeable in CM (versatile too) he is too radical (and dogmatic

) to contribute by himself in this area! The combination of these two will be the ideal chemistry we can pray for!).

The technology and tools are all currently available! It is only the question of forging them together is what is needed! My friend Subramanian has already done all the fundamental work from the days of Quick basic'! He did not push further due to age and other commitments. He ought to have been honoured by MA with an award for 'High Tech Contribution to CM'. His fundamental work on modelling gamakas are classics ( mostly SSP based). But his techniques are crude and have to be modernized. If I were 20 years younger I would single-handedly have forged ahead. Perhaps I am crying in the wilderness

But I am not quitting! Only trying to identify folks with a similar passion for innovations in 'CM Technology' through these threads!

end of rant!

cmlover · Post by **cmlover** » 19 Apr 2006, 20:49

parenthetically let me add that Digital Ear would have long implemented the project if there was money in it! There is no money in CM ! Only Atma tripti

arunk · Post by **arunk** » 19 Apr 2006, 21:32

cmlover,

I hear you! I would like to spend more time but i havent had as much time

. But I have not quite faded out completely from the "technical" side of CM, but must admit that i havent had the time and energy to see my ideas through into something tangible and more useful and that is a major failing

I am of course now more into the practical side of cm (as in learning it), which I was a full zero before. My hope is to feed that back into the technical side. So its not like my involvement with cm has diminished at all, its more like i am not able juggle all things effectively in the little spare time i have during a normal week! I do regret that time and again. I need a few million dollars so that i can quit working and do this (and vacation) more

. But dont we all!!

If it is any consolation, even right now, I am dabbling with a new "project". This midi synthesis stuff was the prior "project" (and i will conveniently blame a computer crash as a catalyst for "jumping off" of that). Of course the one before that was the first one - pitch detection. There is definitely a possible marriage between the pitch detection and the synthesis stuff as you guys also have found out.

Regards
Arun

kiransurya · Post by **kiransurya** » 19 Apr 2006, 21:44

Very complicated technology

(no offense please!!!!!). My head is spinning.......

cmlover · Post by **cmlover** » 20 Apr 2006, 00:04

vk

Just wanted to give you an idea of what Arun had achieved.
Here is the famous phrase in Abheri ever
http://rapidshare.de/files/18425547/janaki44.wav.html
here is what Arun had achieved three years ago through his synthesis engine
http://rapidshare.de/files/18425680/jan ... h.wav.html
here is what Digital ear is able to do
http://rapidshare.de/files/18425759/janaki44.mid.html

The gamakam capsulation is indeed not more difficult. Actually Arun's synthetic approach permits one to attck the gamakam issue at the fundamenatal 'atomic' level. One can manipulate analytically and weigt the harmonics to twist and turn the waves microscopically whereas the Digital ear is a 'dumb' approximation 'curve fitting' approach.

I can only hope that someday he will find time to work on it! If I had a million I would gladly share it with him

arunk · Post by **arunk** » 20 Apr 2006, 00:47

cmlover/vk.

That synthesis i did was more to validate my pitch detection. Calling it a "synthesis engine" is an overstatement - as it was quite dumb. The intelligence is in the pitch detector and synthesis part merely crunched the "core" data given to by the pitch detector. I imagine Digital ear has a pitch detector too (isnt that right?).

Pitch detection gets funky when you throw in polyphony (lets say there was a violin playing along with janaki in a audibly perceptible way) or even certain male voices. So doing it say on concert renditions can have mixed results. But should be pretty ok for individual singing (as long as tampura is not too loud)

The synthesis approach i used (additive synthesis) if stars aligned right can "replicate" the timbre (tonal character) of the original sound as maybe would be obvious from the synthesized sample. Also, *theoretically*, assuming you could build realistic timbre models from the data, you should be able to say you could change janaki's voice to MDR's and hear him sing singAravElanE in janaki's version

Now what a contrast that would be - lol!!!

But here Digital ear does a very very good job. But if I remember right, it doesnt always - i.e. there were cases where my pitch detector detected the pitches cleaner than what Digital ear's reproduction indicated. This is generally true for almost all pitch detectors - atleast the ones that are commercially available.

My last foray into synthesis part involved synthesizing (those curves i talked) using MIDI only and so is not additive synthesis but more like Digital ear from a synthesis standpoint.

Arun

cmlover · Post by **cmlover** » 20 Apr 2006, 01:42

Arun/vk/others

Here is a different question. Laya should be easier to machine implement. The rhythms can be prepogrammed, The number of taLas are limited and even the 'nadais' are mostly standard. I have seen limited success with the 'dumb' drum machines which sound terrible in the CM context. But it is easy to implement a realtime detection of notes and the rhythm and adjust the play of the mridangam(I mean machine-play) appropriately! Even sensing the shadjam and panjamam would produce a reasonable alignment of the rhythm. Now that our cyber vidyalaya (I am sure you will be there!) is starting we can think of semi-automating laya accompaniment!

Your views please!

arunk · Post by **arunk** » 20 Apr 2006, 02:17

cmlover,

realtime detection of rhythmic beats is not that easy. It has its own complexities (if i remember correctly what I read in many tech articles ).

But you dont need that to come up with something that could be useful. I think mridangam has a finite number of basic syllables (tha dhim thom etc.) which translate to specific sounds. That is your base sound bank. You should able to build basic building block rythmic patterns in terms using sollukattus (?right terminology? the one used by konakkol), These are really (or translate to) a "string" of the mridangam syllables and hence can be synthesized. Each such pattern of course has a precise duration in terms of mathrais. Each mathrai takes "n" units of time, depending on MIDI tempo. So in theory you could allow people to program the mridangam patern. You could also come up with something sophisticated where the computer "intelligently" picks varying basic building blocks that add up to "n" mathrais etc.

(ps: whether this would fly or not i dont know, but this was another "blue sky" idea i had!)

Must emphasize, my knowledge of mridangam is zero and I do realize that I am presenting a very very gross simplification. I also apologize for erroneous assumptions too. I do know that mridangam playing has a lot lot more intelligence to it!!! The extemporious nature of it is what adds colour to the performance. But it may be possible to come up something that is possible (and passable) for accompanying amateurs and home-grown enthusiasts.

Besides Radel has something like this for Hindustani and not for carnatic. I asked my contact why and the reasons were similar to what I indicate above (i.e. too extemporious for people to buy into it)

Arun

cmlover · Post by **cmlover** » 20 Apr 2006, 03:25

Arun

You have the basic ideas. HM rhytms are supposed to be much simpler than CM. I am waiting to pick up more details once Param starts the discussions!

Most often the vocalist deviates from the rhythm and the mridangist tries to adjust himself! Otherwise he/she will be fired

Computers won't!
I once saw a software where complicated rhythms could be pre-programmed (the tom-toms. hihats etc) and could be cued (Is it steinberg's 'Rebirth One' or FruitLoops I am not sure!) Any way worth investigating! Thanks for the interest!

mohan · Post by **mohan** » 20 Apr 2006, 09:30

I have tried playing using the Radel talameter and it is a good aid for practice but it is very hard to adjust to in a concert situation.

I have also tried putting the tala meter againist some recordings of the masters and half way through the song the talameter and the recording are out of sync.

vasanthakokilam · Post by **vasanthakokilam** » 20 Apr 2006, 10:03

Arun,

That is fantastic. ( Thanks CML for digging that out and also providing the comparision with Digital Ear ). Definitely much better at synthesis with timbre and tonal character which the music reco engine does not do. Great work.

So, the Janaki voice was synthesized by additive synthesis to come up with the right harmonics? Just making sure I understand your tool correctly. I would like to know more on: what all does your pitch detector do and what is its relationship to the synthesis, if any?

I think what you have is useful for many applications already. As you described, we can play around with different pitch curves and see if that can be used as a new gamakam. It will still be a search like a needle in the haystack and but it is a worthwhile endeavor.

Arvindh's work fits in nicely with what you had already done. His melodic units provide the necessary closure for all of CM ( paraphrasing him from his paper). Arun, on your next revision to the tool, if you can provide a 'Macro' for some or all of Arvindh's melodic units, then even a person like me can try to code some passages in this macro language for a few ragas and see how it sounds.

CML, can you explain more about your friend Subramaniyan's work? I think I remember listening to an item produced with his program that you posted at the BB. It sounded very nice with all the gamakams.

arunk · Post by **arunk** » 20 Apr 2006, 20:00

vk,

So, the Janaki voice was synthesized by additive synthesis to come up with the right harmonics? Just making sure I understand your tool correctly. I would like to know more on: what all does your pitch detector do and what is its relationship to the synthesis, if any?

disclaimer: i am an amateur in this and so i would suggest not taking what i say for granted but double-checking it. I am no DSP or acoustic expert.

Yes sort of. What the pitch detector does is analyzes a sound sample to extract the "fundamental frequency" and harmonics (e.g. if you sing sa exactly at a pitch of C i.e. 1-kattai, then your voice has a many frequency components. One at fundamental = 130Hz, and harmonics around 260, 390, etc. ), and their respective amplitudes. ).

The distribution of the amplitude across these components contributes to the tonal character. Once you have a good estimate of this you can use additive synthesis and reproduce the voice or original signal source. Most pitch detectors need only the fundamental to synthesize (say if you want to do midi), but to build a realistic timbre model (note: i have no clue how!) i would think you need this and many such data points i.e. for various pitches (and i would imagine even for various sounds if you want to replicate sahityam - this is my guess)

Also when i say "extract" above, i mean that the pitch detector ignores other frequency components in the original signal (say noise, mridangam, violin/tampura if low-enough), which it can do so reasonably well if the other components are not "strong enough" to confuse it. But this is where there is a large potential for error. Accurate, reliable pitch detection is very hard to almost impossible. First, there is always margin of error but on a good success rate (say from a good clean signal) the error would not be audibly perceptible. Second, you can go wrong in pitch value itself, as the "extraneous" components can easily confuse it depending on how they appear in the original signal.

I have analyzed voice, flute, guitar, veena and have had fair results. See http://www.geocities.com/arunk_15/pitch ... oject.html for a couple of examples - the website content is quite old.

But if you have an instrument that has multiple strings that can be played at same time (violin in CM, western guitar playing chords), my algorithm would go bonkers - but that is because of polyphony. Polyphonic pitch detection is extremely hard. You pretty much have to detects all components for all pitches - without getting confused by extraneous components. Also if you have an old recording of male voices, things get hard to. But if I record a singer singing at home, it usually does a fair job (again, as long as tampura is not too loud to confuse it).

Arun

arunk · Post by **arunk** » 20 Apr 2006, 20:10

His melodic units provide the necessary closure for all of CM ( paraphrasing him from his paper). Arun, on your next revision to the tool, if you can provide a 'Macro' for some or all of Arvindh's melodic units, then even a person like me can try to code some passages in this macro language for a few ragas and see how it sounds.

Well that was sort of the intent - although i was first trying basic stuff to make sure it was viable and then i wanted to try arvindh's stuff. I wasnt getting good results even for the first one i tried, not because the program was flawed but i wasnt being patient. But I was also concentrating more on the programming side (i.e. adding features to the program so it made my experimentation easier), and then the computer crashed ... By the time I fixed, i had this "other" "bright" idea and it was time to jump ship!

As I mentioned I will definitely try to resurrect it. I checked its state yesterday and it seems to be in decent shape that with a few tweaks it may be usable by someone other than me for basic experimentation. I will keep you posted but i may need some prodding

I have had a bad history of leaving things half-cooked.

Arun

cmlover · Post by **cmlover** » 20 Apr 2006, 21:40

The baby is now a toddler! You must have more time now! Also try to loop in other bright (CM loving) youngsters to collaborate with you! It is a labour of love! We can brainstorm ideas at this forum...

arunk · Post by **arunk** » 30 Apr 2006, 19:50

Vk/cmlover,

An update. I am working on "touching up" (although in truth it is more than a touch up) of that java program i wrote that allows you to construct arbitrary pitch curves and synthesize them in MIDI. I hope to put a prelim version on my web in a few days.

Arun

cmlover · Post by **cmlover** » 30 Apr 2006, 21:07

Attaboy!
Awaiting the release!
Please give us the link to watch for!
Will also be a willing beta tester...(no java experience

)