Carnatic Youtube Collection

Miscellaneous topics on Carnatic music
Post Reply
ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#1 Carnatic Youtube Collection

Post by ranjanimalavi »

Carnatic Youtube Collection:

https://ramanarunachalam.github.io/Musi ... natic.html

I have aggregated (crawling youtube) around 37,000 youtube videos for my listening pleasure.
I do have more, will be enable as i fix the noises.

Note that, its still in the making, so there are few issues:

Search doesn't work, as its hosted statically.

You can click on the PLAY, it will play if its the first, otherwise adds to the playlist.
The button left of search you can click to delete items in the play list.

Hope its useful to somebody

Raman

On a SIde Note, if you are a Bangalorean, and like to know the trees in Lalbagh or Cubbon park

you can follow the links for my Tree Logger App:

https://sites.google.com/site/ramanarunachalamhome

Lakshman
Posts: 13284
Joined: 10 Feb 2010, 18:52
x 34

#2 Re: Carnatic Youtube Collection

Post by Lakshman »

You have listed under Dandayaudapani Pillai the song Ayiram Ayiram. The song is a composition of Neela Ramamurthi.

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#3 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Thanks Lakshman ji, corrected it.

Raman

SrinathK
Posts: 2428
Joined: 13 Jan 2013, 16:10
x 186
x 40

#4 Re: Carnatic Youtube Collection

Post by SrinathK »

Please add this in Internet resources section also. This collection compiled from YT is absolutely massive!

Pratyaksham Bala
Posts: 3863
Joined: 21 May 2010, 16:57
x 291
x 21

#5 Re: Carnatic Youtube Collection

Post by Pratyaksham Bala »

@ranjanimalavi
Thank you !
Admirable contribution !

musicofmdr
Posts: 22
Joined: 16 May 2020, 18:36
x 7
x 5

#6 Re: Carnatic Youtube Collection

Post by musicofmdr »

@ranjanimalavi, WoW!!! This is an amazing effort! Thank you very much! Hope it is kept up-to-date as more and more videos gets uploaded.

Many a times, finding a particular item in Youtube has been challenging. This indexed collection is going to be a great time saver!

#musicofmdr
https://musicofmdr.com

Nick H
Posts: 9209
Joined: 03 Feb 2010, 02:03
x 1015
x 36

#7 Re: Carnatic Youtube Collection

Post by Nick H »

This seems to me to be achieving the impossible!

Not only indexing the videos on youtube, which youtube/google can do without a problem, but going to the songs within the videos? Massive work, not only in the programming, but in the listening and listing. Honestly don't know how it could even be attempted.

Amazing.

vgovindan
Posts: 1859
Joined: 07 Nov 2010, 20:01
x 92
x 14

#8 Re: Carnatic Youtube Collection

Post by vgovindan »

A fantastic effort. God bless.

cienu
Posts: 2368
Joined: 04 Feb 2010, 11:40
x 368
x 12

#9 Re: Carnatic Youtube Collection

Post by cienu »

How did you do this ? Such a treasure . My compliments to you

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#10 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

I added for curiosity the youtube video time, here are the top ones

Total : 17536 hours
M. Balamurali Krishna : 443 hours
T. M. Krishna : 304 hours
Sanjay Subramanian : 265 hours
M. S. Subbulakshmi : 264 hours
Lalgudi Jayaraman : 233 hours
Semmangudi Srinivasa Iyer : 228 hours
M. D. Ramanathan : 221 hours
M. S. Gopalakrishnan : 174 hours
T. N. Seshagopalan : 170 hours
Sudha Raghunathan : 163 hours
M. L. Vasanthakumari : 163 hours
K. V. Narayanaswamy : 151 hours
Mandolin U. Srinivas : 147 hours
Maharajapuram Santhanam : 139 hours
M. Chandrasekaran : 137 hours
S. Sowmya : 135 hours
Madurai Mani Iyer : 126 hours
T. N. Krishnan : 123 hours
G. N. Balasubramaniam : 121 hours
D. K. Jayaraman : 121 hours
C. Saroja and C. Lalitha : 118 hours
Ramakrishna Murthy : 118 hours
Nedunuri Krishnamurthy : 110 hours
Nithyasree Mahadevan : 107 hours
D. K. Pattammal : 105 hours
T. R. Mahalingam : 98 hours
Abhishek Raghuram : 98 hours
K. J. Yesudas : 98 hours
Bombay Jayshree : 96 hours
Prince Rama Varma : 94 hours
Ranjani and Gayathri : 90 hours

SrinathK
Posts: 2428
Joined: 13 Jan 2013, 16:10
x 186
x 40

#11 Re: Carnatic Youtube Collection

Post by SrinathK »

Please crawl the channels of shankar rajashekaran, raju ashokan and T V Gopalakrishnan, the Big 3 of sangeethapriya and CM YT.

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#12 Re: Carnatic Youtube Collection

Post by Sath »

Amazing indexing and very valuable to the Carnatic world!

I'm curious as I'm sure some others are. Is at least some of the identification done through manual listening?

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#13 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Its purely based on the meta data: title and description coming part of search,
not even through full description when you play the video.

it does introduce noise, which can be filtered over time.

Some of the issues are like two artists mentioned not related to each other, will be in both.
MMI singing Ravichandrika in TNR and MMI, because of the metadata mentioning both.

very difficult to identify composer and artist if the song is not known and he is singing it.
GNB doesn't sing his composition except one, but Lalgudi and MDR are problems.
Sishya parampara's are problems, centenary concerts are problems.

5% error is inevitable, as the data is not perfect.

Raman

musicofmdr
Posts: 22
Joined: 16 May 2020, 18:36
x 7
x 5

#14 Re: Carnatic Youtube Collection

Post by musicofmdr »

ranjanimalavi wrote: 08 Jun 2020, 09:20 Its purely based on the meta data: title and description coming part of search,
not even through full description when you play the video.

it does introduce noise, which can be filtered over time.
...
Raman
Is it possible to accept input from other rasikas? (something like crowd sourcing?) This will help to filter out the noise and also speed up addition of more uploads into the crawled list (especially those which do not have enough/right metadata). If a listener finds out that a particular youtube link is not yet added, it would be great to have an option to suggest inclusion of this link to the DB (with additional metadata, if applicable).

#musicofmdr
https://musicofmdr.com/

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#15 Re: Carnatic Youtube Collection

Post by Sath »

Thanks ranjanimalavi. That sounds like an excellent approach. Good to know the limitations which are minimal.

How do you account for variations in spelling?

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#16 Re: Carnatic Youtube Collection

Post by Sath »

Some more questions:
1) 0404 - Thillana - gidhu nadiku takiTa takadhrka Duta tOm : Jhampa - Swathi Thirunal - Mandolin U. Srinivas USRINIVAS01 points to https://www.youtube.com/watch?v=h3ZZENjQk60 The video title is simply "Mandolin U Srinivas-Tillana-Dhanashri-Swati Tirunal"

How does it know the full title of the song? Is it assuming that there is only one thillana that Swathi Thirunal composed in Dhanashri?

2) Is it possible to include youtube links as an alternate to the player, since I use my own youtube handler. Plus, on my phone, the website's built in player doesn't work.

3) Is this open source?

4) Are you considering making an Android app by any chance?

5) Just a suggestion: is it really useful to have raga tuples (eg: "Dhanashree,Natabhairavi") indexed, or does it make sense to have these tracks appear under both ragas separately? My two cents: the raga combinations makes the raga index hard to use.

Again, this is super awesome. I listen all the time to carnatic music on youtube, and it's incredible to be able to discover music by compositions, composers, and so on. Thanks a ton for making and sharing this!

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#17 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Suggestions are always welcome, I will try to implement it as much as possible.
I am still fixing various issues which bothers me, so I have better experience.
Hopefully, by end of month it will become better. SO I can enhance with new videos.
I am not a UI person, so if its simple then I can do.

I am validating everything against a collection built over the years, which is what this site used to point to.
Since many in that were copy righted, i didn't upload any song.

I use the same collection to catch spelling errors. I do generate various phrases.
Dikshithar - gives 100's of variation in spelling - people also write it as Dixitar

I have collected for the songs the lyrics, so it plugs in the first 50 characters of the pallavi.

this whole things is on GIT, anybody can take it and use it.

I am thinking of making an Android App. I am not an UI guy.
Similar to the Tree Logger App I have.
These are built using PYTHON including my App

The multiple Raga - some times there are 2, 3, 4 raga, RTP for slokham/virutham i can replace it with Ragamalika

Raman

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#18 Re: Carnatic Youtube Collection

Post by Sath »

Great, thank you for the response.

Is it possible to have each row have a youtube link (in addition to the play button?) This'll be so much more helpful at least for me.

I mean: is the code that indexes and generates your website open source? If so, a link would be most appreciated.

Ragamalika is a good idea. However, it is valuable to know the ragas in it. One idea is to prepend 'Ragamalija' so all Ragamalikas are grouped together in the sorting. Eg:

"Dhanashree,Natabhairavi" would become:
"Ragamalika,Dhanashree,Natabhairavi"

There are probably even better ways. I'll try to think of some and post later.

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#19 Re: Carnatic Youtube Collection

Post by Sath »

Any plans or chance in making the raw database available so one can query directly into it?

vsn69
Posts: 132
Joined: 02 Oct 2017, 17:30
x 26

#20 Re: Carnatic Youtube Collection

Post by vsn69 »

Amazing collection @rajanimalavi
Appreciate your work very, very much.

See if you can these as well
https://www.youtube.com/playlist?list=P ... zU6Idn-ygJ

She's Smt Rajam Pushpavanam, daughter of Madurai Pushpavanam and cousin of my Madurai Mani Iyer.
And my mother.
Thanks and best regards,

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#21 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Now, the carnatic youtbe video count stands around 66,500+
Fixed many errors, and removed lots of film videos sneaked in because of some of the artists sing film songs.
Also added transliteration for 6 languages: this needs more work, over time it will get fixed.

@vsn69 - i have added Smt Rajam Pushpavanam

Raman

vsn69
Posts: 132
Joined: 02 Oct 2017, 17:30
x 26

#22 Re: Carnatic Youtube Collection

Post by vsn69 »

ranjanimalavi wrote: 27 Aug 2020, 21:28 Now, the carnatic youtbe video count stands around 66,500+
Fixed many errors, and removed lots of film videos sneaked in because of some of the artists sing film songs.
Also added transliteration for 6 languages: this needs more work, over time it will get fixed.

@vsn69 - i have added Smt Rajam Pushpavanam

Raman
Truly great work you are doing.
And thank you.

sankark
Posts: 2046
Joined: 16 Dec 2008, 09:10
x 105
x 40

#23 Re: Carnatic Youtube Collection

Post by sankark »

Amazing & a valuable contribution.

Wondering: did you write an RPA BOT to scrape YT and build this index?

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#24 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

No, Crawled using pyppeteer a python package

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#25 Re: Carnatic Youtube Collection

Post by Sath »

Any plans or chance in making the raw database available so one can query directly into it?

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#26 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

when i move from Git to a server, i will do that.
these are statically hosted pages

S.M. Vasudevan
Posts: 20
Joined: 20 Jun 2020, 06:48
x 2
x 5

#27 Re: Carnatic Youtube Collection

Post by S.M. Vasudevan »

Hello. I have posted a Madurai Mani Iyer rendition of Sarasa Sama Dana. It is from what I have seen the best quality rendition. I have made the Audio better and digitized it. It is very special with he elaboration and length. Here is the link: https://www.youtube.com/watch?v=Y_B_JZpUi1c&t=414s

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#28 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Its more than 71K videos.
I have added a search box, which will work phonetically in most cases unless a word is broken.
Corrected many Tamil Lyrics as the rest of the languages are little easier.

Sath,
I converted all data to json, see if you can use it as it is.

Raman

shankarank
Posts: 3676
Joined: 15 Jun 2009, 07:16
x 5
x 34

#29 Re: Carnatic Youtube Collection

Post by shankarank »

And to process json here is a command line language.

https://stedolan.github.io/jq/download/

Everything is done on the fly, entire soup to nuts by chaining filters and aggregates.

Databases like postgre / mysql also support turning JSON content into a projected table, but the above is powerful , don't have to load it anywhere.

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#30 Re: Carnatic Youtube Collection

Post by Sath »

Json would be fantastic. Raman: could you please point me to the json? I'm well versed in using it and so on, just need a url or other means to access. Thanks so much again, this is terrific!

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#31 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

My Site: https://ramanarunachalam.github.io/Musi ... natic.html

Everything else is in github only: https://github.com/ramanarunachalam/Mus ... r/Carnatic

whoever wants can design their own website based on the json data

Raman

Ananth
Posts: 133
Joined: 14 Feb 2006, 14:04
x 30

#32 Re: Carnatic Youtube Collection

Post by Ananth »

Fantastic effort, ranjanimalavi!

Some thoughts, purely as a feedback.

1. Spellings seem to be handled well. I find 3 versions of endaro mahanubhavulu by Trichur Brothers listed together!

2. I think dependence on YouTube Metadata is too constraining. But I have noticed something. After listening to a sing in Arabhi which does not even mention raga name anywhere, I'm recommended more arabhi songs (even devagandhari songs!) How does this work?

3. As someone mentioned, crowdsourcing Metadata can be tried. I see such information in comments often, but requires some standard approach if a bot has to pull it.

4. How about annotating the songs in full concerts? Any thoughts?

Thanks again!


BTW is your name a portmanteau of your two favorite ragas?😀

MaheshS
Posts: 1181
Joined: 02 Feb 2010, 22:36
x 101
x 1

#33 Re: Carnatic Youtube Collection

Post by MaheshS »

This is going viral in a Whatsapp forward today, I have received the same message about 10 times from various different persons!

Code: Select all

This is a directory of 37,000 Youtube videos of Carnatic Music, searchable by Raga/Artiste/Composer/Kriti. A Bangalorean has aggregated this by crawling though Youtube. Some interesting data from him on this collection :

Total : 17536 hours
M. Balamurali Krishna : 443 hours
T. M. Krishna : 304 hours
Sanjay Subramanian : 265 hours
M. S. Subbulakshmi : 264 hours
Lalgudi Jayaraman : 233 hours
Semmangudi Srinivasa Iyer : 228 hours
M. D. Ramanathan : 221 hours
M. S. Gopalakrishnan : 174 hours
T. N. Seshagopalan : 170 hours
Sudha Raghunathan : 163 hours
M. L. Vasanthakumari : 163 hours
K. V. Narayanaswamy : 151 hours
Mandolin U. Srinivas : 147 hours
Maharajapuram Santhanam : 139 hours
M. Chandrasekaran : 137 hours
S. Sowmya : 135 hours
Madurai Mani Iyer : 126 hours
T. N. Krishnan : 123 hours
G. N. Balasubramaniam : 121 hours
D. K. Jayaraman : 121 hours
C. Saroja and C. Lalitha : 118 hours
Ramakrishna Murthy : 118 hours
Nedunuri Krishnamurthy : 110 hours
Nithyasree Mahadevan : 107 hours
D. K. Pattammal : 105 hours
T. R. Mahalingam : 98 hours
Abhishek Raghuram : 98 hours
K. J. Yesudas : 98 hours
Bombay Jayshree : 96 hours
Prince Rama Varma : 94 hours
Ranjani and Gayathri : 90 hours
https://ramanarunachalam.github.io/Music/Carnatic/carnatic.html
Fame at last ranjanimalavi? :D

SrinathK
Posts: 2428
Joined: 13 Jan 2013, 16:10
x 186
x 40

#34 Re: Carnatic Youtube Collection

Post by SrinathK »

And on FB carnatic groups too. It's a copy of the post from this thread.

shankarank
Posts: 3676
Joined: 15 Jun 2009, 07:16
x 5
x 34

#35 Re: Carnatic Youtube Collection

Post by shankarank »

Would this have made into this time capsule?

https://bigthink.com/technology-innovat ... belltitem2

What good is open source software code, which gets obsolete in 10 years or 20 years?

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#36 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

FYI: https://ramanarunachalam.github.io/Musi ... natic.html

The video count is now at 83K +

Now you can view the site in all 6 languages completely:
English, Tamil, Telugu, Kannada, Malayalam, Sanskrit(Devanagari)

Please feel free to comment on the errors in the transliteration.

Raman

Sachi_R
Posts: 1942
Joined: 31 Jan 2017, 20:20
x 59
x 28

#37 Re: Carnatic Youtube Collection

Post by Sachi_R »

Dear Raman,
Congratulations on a great initiative. Shows your love for CM in action.

I always believed Ranjani Malavi were two ladies.

By the way I tried to use your site just now. Blame it on Sunday morning blues or what But after it made a playlist I didn't know how to play the video.

Just curious - how did you timestamp the links to the song in question in the middle of a long Youtube video? I know how you might have done it but It would have taken you lots of effort to mark the times and generate the Urls.... God bless.

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#38 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Ranjani/Malavi are my daughters.

You can click on the playlist (icon with lines) and add/modify it.
Some videos don't play, then you have delete it, or it will ask you to open in youtube app itself.

I might not have got the timestamped ones.
If you use youtube-dl it gives you each part/part

If its chrome on mobile (choosing the corresponding language (say tamil, kannada etc), the microphone works very well).
Desktop microphone on chrome works for smaller words are if you speak faster.
Also i have given keyboard to enter into any one of the 6 languages.
For some words tamil will be a problem, as its not phonetic.

My joy in doing this was i like to see things in Indic Languages.

Raman

Sachi_R
Posts: 1942
Joined: 31 Jan 2017, 20:20
x 59
x 28

#39 Re: Carnatic Youtube Collection

Post by Sachi_R »

Great.

Indian languages definitely need a lot of support. It is so much nicer to hear someone speak about Carnatic music in Kannada, for me.

Sath
Posts: 15
Joined: 10 Jan 2020, 12:37
x 1

#40 Re: Carnatic Youtube Collection

Post by Sath »

ranjanimalavi wrote: 30 Oct 2020, 14:43 My Site: https://ramanarunachalam.github.io/Musi ... natic.html

Everything else is in github only: https://github.com/ramanarunachalam/Mus ... r/Carnatic

whoever wants can design their own website based on the json data

Raman
Just wanted to say again what a fantastic contribution this is! I've been using it all the time for several months. Thanks also for making the json sources available too.

msakella
Posts: 2072
Joined: 30 Sep 2006, 21:16
x 7

#41 Re: Carnatic Youtube Collection

Post by msakella »

A very great and excellent work you did which helps a lot to our poor aspirants. Had there been any other person like you preserving such videos bringing out the rare finger techniques of our giant Violinists for educational purposes it would have helped many violin aspirants a lot. But, in the absence of such a noble person, very sadly, not even a single video has ever been preserved and it is a great loss of great treasure to our Indian Music. Our people are so great!!! msakella

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#42 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

I also started a Hindustani Youtube Collection
https://ramanarunachalam.github.io/Musi ... stani.html
This needs more enhancements compared to the carnatic music site.
But consolidated for 50+ top artists, around 20,000+ videos

Carnatic Youtube Collection:
https://ramanarunachalam.github.io/Musi ... natic.html
This has now around 1,10,000+ videos, 200-300 videos aggregated per week

Raman

ganesh_mourthy
Posts: 1305
Joined: 02 Sep 2007, 23:08
x 5
x 11

#43 Re: Carnatic Youtube Collection

Post by ganesh_mourthy »

Extremely thank you !

Purist
Posts: 429
Joined: 13 May 2008, 16:55
x 1
x 2

#44 Re: Carnatic Youtube Collection

Post by Purist »

ranjanimalavi wrote: 17 Jun 2021, 16:51 Carnatic Youtube Collection:
https://ramanarunachalam.github.io/Musi ... natic.html
This has now around 1,10,000+ videos, 200-300 videos aggregated per week

Raman
Doesn't play the selected audio/video when clicked. Is there a login process.
How to play the selected audio/video after selecting the artist?

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#45 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

There is no login

You have to click on the play button.
SOme videos are restricted by some users, it will say it can be played in youtube site only or it could have been deleted.
in those case you have click the playlist and delete them, or select them and play in youtube site itself.

Raman

thanjavooran
Posts: 2759
Joined: 03 Feb 2010, 04:44
x 259
x 16

#46 Re: Carnatic Youtube Collection

Post by thanjavooran »

Ranjanimalavi. Avl
Many thanks
Best wishes.
Thanjavooran
22 06 2021

ranjanimalavi
Posts: 282
Joined: 17 Feb 2007, 06:15
x 9

#47 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

I have added Youtube icon external link for playing in separate tab or in youtube application on phone.
You can right click and copy the link also.
Otherwise many people are confused about the embedded youtube iframe, which is based on the global playlist,
which you need to edit if the video is not allowed to play or removed.

Updated Hindustani Youtube Collection
https://ramanarunachalam.github.io/Musi ... stani.html
Updated with 60,000+ videos, 340+ Artists, 550 Ragas

This needs more enhancements compared to the carnatic music site with respect to songs:
If i can find top banishes, i can add them. As I am not aware of them not able to add it.

Carnatic Youtube Collection:
https://ramanarunachalam.github.io/Musi ... natic.html
This has now around 1,11,000+ videos, 200-300 videos added per week

Raman

Post Reply