Carnatic Youtube Collection

Miscellaneous topics on Carnatic music
Post Reply
ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#1 Carnatic Youtube Collection

Post by ranjanimalavi »

Carnatic Youtube Collection:

https://ramanarunachalam.github.io/Musi ... natic.html

I have aggregated (crawling youtube) around 37,000 youtube videos for my listening pleasure.
I do have more, will be enable as i fix the noises.

Note that, its still in the making, so there are few issues:

Search doesn't work, as its hosted statically.

You can click on the PLAY, it will play if its the first, otherwise adds to the playlist.
The button left of search you can click to delete items in the play list.

Hope its useful to somebody

Raman

On a SIde Note, if you are a Bangalorean, and like to know the trees in Lalbagh or Cubbon park

you can follow the links for my Tree Logger App:

https://sites.google.com/site/ramanarunachalamhome

Lakshman
Posts: 12621
Joined: 10 Feb 2010, 18:52
x 19

#2 Re: Carnatic Youtube Collection

Post by Lakshman »

You have listed under Dandayaudapani Pillai the song Ayiram Ayiram. The song is a composition of Neela Ramamurthi.

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#3 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Thanks Lakshman ji, corrected it.

Raman

SrinathK
Posts: 2304
Joined: 13 Jan 2013, 16:10
x 170
x 16

#4 Re: Carnatic Youtube Collection

Post by SrinathK »

Please add this in Internet resources section also. This collection compiled from YT is absolutely massive!

Pratyaksham Bala
Posts: 3807
Joined: 21 May 2010, 16:57
x 260
x 16

#5 Re: Carnatic Youtube Collection

Post by Pratyaksham Bala »

@ranjanimalavi
Thank you !
Admirable contribution !

musicofmdr
Posts: 20
Joined: 16 May 2020, 18:36
x 7
x 5

#6 Re: Carnatic Youtube Collection

Post by musicofmdr »

@ranjanimalavi, WoW!!! This is an amazing effort! Thank you very much! Hope it is kept up-to-date as more and more videos gets uploaded.

Many a times, finding a particular item in Youtube has been challenging. This indexed collection is going to be a great time saver!

#musicofmdr
https://musicofmdr.com

Nick H
Posts: 9138
Joined: 03 Feb 2010, 02:03
x 921
x 26

#7 Re: Carnatic Youtube Collection

Post by Nick H »

This seems to me to be achieving the impossible!

Not only indexing the videos on youtube, which youtube/google can do without a problem, but going to the songs within the videos? Massive work, not only in the programming, but in the listening and listing. Honestly don't know how it could even be attempted.

Amazing.

vgovindan
Posts: 1859
Joined: 07 Nov 2010, 20:01
x 92
x 14

#8 Re: Carnatic Youtube Collection

Post by vgovindan »

A fantastic effort. God bless.

cienu
Posts: 2361
Joined: 04 Feb 2010, 11:40
x 352
x 7

#9 Re: Carnatic Youtube Collection

Post by cienu »

How did you do this ? Such a treasure . My compliments to you

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#10 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

I added for curiosity the youtube video time, here are the top ones

Total : 17536 hours
M. Balamurali Krishna : 443 hours
T. M. Krishna : 304 hours
Sanjay Subramanian : 265 hours
M. S. Subbulakshmi : 264 hours
Lalgudi Jayaraman : 233 hours
Semmangudi Srinivasa Iyer : 228 hours
M. D. Ramanathan : 221 hours
M. S. Gopalakrishnan : 174 hours
T. N. Seshagopalan : 170 hours
Sudha Raghunathan : 163 hours
M. L. Vasanthakumari : 163 hours
K. V. Narayanaswamy : 151 hours
Mandolin U. Srinivas : 147 hours
Maharajapuram Santhanam : 139 hours
M. Chandrasekaran : 137 hours
S. Sowmya : 135 hours
Madurai Mani Iyer : 126 hours
T. N. Krishnan : 123 hours
G. N. Balasubramaniam : 121 hours
D. K. Jayaraman : 121 hours
C. Saroja and C. Lalitha : 118 hours
Ramakrishna Murthy : 118 hours
Nedunuri Krishnamurthy : 110 hours
Nithyasree Mahadevan : 107 hours
D. K. Pattammal : 105 hours
T. R. Mahalingam : 98 hours
Abhishek Raghuram : 98 hours
K. J. Yesudas : 98 hours
Bombay Jayshree : 96 hours
Prince Rama Varma : 94 hours
Ranjani and Gayathri : 90 hours

SrinathK
Posts: 2304
Joined: 13 Jan 2013, 16:10
x 170
x 16

#11 Re: Carnatic Youtube Collection

Post by SrinathK »

Please crawl the channels of shankar rajashekaran, raju ashokan and T V Gopalakrishnan, the Big 3 of sangeethapriya and CM YT.

Sath
Posts: 13
Joined: 10 Jan 2020, 12:37
x 1

#12 Re: Carnatic Youtube Collection

Post by Sath »

Amazing indexing and very valuable to the Carnatic world!

I'm curious as I'm sure some others are. Is at least some of the identification done through manual listening?

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#13 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Its purely based on the meta data: title and description coming part of search,
not even through full description when you play the video.

it does introduce noise, which can be filtered over time.

Some of the issues are like two artists mentioned not related to each other, will be in both.
MMI singing Ravichandrika in TNR and MMI, because of the metadata mentioning both.

very difficult to identify composer and artist if the song is not known and he is singing it.
GNB doesn't sing his composition except one, but Lalgudi and MDR are problems.
Sishya parampara's are problems, centenary concerts are problems.

5% error is inevitable, as the data is not perfect.

Raman

musicofmdr
Posts: 20
Joined: 16 May 2020, 18:36
x 7
x 5

#14 Re: Carnatic Youtube Collection

Post by musicofmdr »

ranjanimalavi wrote: 08 Jun 2020, 09:20 Its purely based on the meta data: title and description coming part of search,
not even through full description when you play the video.

it does introduce noise, which can be filtered over time.
...
Raman
Is it possible to accept input from other rasikas? (something like crowd sourcing?) This will help to filter out the noise and also speed up addition of more uploads into the crawled list (especially those which do not have enough/right metadata). If a listener finds out that a particular youtube link is not yet added, it would be great to have an option to suggest inclusion of this link to the DB (with additional metadata, if applicable).

#musicofmdr
https://musicofmdr.com/

Sath
Posts: 13
Joined: 10 Jan 2020, 12:37
x 1

#15 Re: Carnatic Youtube Collection

Post by Sath »

Thanks ranjanimalavi. That sounds like an excellent approach. Good to know the limitations which are minimal.

How do you account for variations in spelling?

Sath
Posts: 13
Joined: 10 Jan 2020, 12:37
x 1

#16 Re: Carnatic Youtube Collection

Post by Sath »

Some more questions:
1) 0404 - Thillana - gidhu nadiku takiTa takadhrka Duta tOm : Jhampa - Swathi Thirunal - Mandolin U. Srinivas USRINIVAS01 points to https://www.youtube.com/watch?v=h3ZZENjQk60 The video title is simply "Mandolin U Srinivas-Tillana-Dhanashri-Swati Tirunal"

How does it know the full title of the song? Is it assuming that there is only one thillana that Swathi Thirunal composed in Dhanashri?

2) Is it possible to include youtube links as an alternate to the player, since I use my own youtube handler. Plus, on my phone, the website's built in player doesn't work.

3) Is this open source?

4) Are you considering making an Android app by any chance?

5) Just a suggestion: is it really useful to have raga tuples (eg: "Dhanashree,Natabhairavi") indexed, or does it make sense to have these tracks appear under both ragas separately? My two cents: the raga combinations makes the raga index hard to use.

Again, this is super awesome. I listen all the time to carnatic music on youtube, and it's incredible to be able to discover music by compositions, composers, and so on. Thanks a ton for making and sharing this!

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#17 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Suggestions are always welcome, I will try to implement it as much as possible.
I am still fixing various issues which bothers me, so I have better experience.
Hopefully, by end of month it will become better. SO I can enhance with new videos.
I am not a UI person, so if its simple then I can do.

I am validating everything against a collection built over the years, which is what this site used to point to.
Since many in that were copy righted, i didn't upload any song.

I use the same collection to catch spelling errors. I do generate various phrases.
Dikshithar - gives 100's of variation in spelling - people also write it as Dixitar

I have collected for the songs the lyrics, so it plugs in the first 50 characters of the pallavi.

this whole things is on GIT, anybody can take it and use it.

I am thinking of making an Android App. I am not an UI guy.
Similar to the Tree Logger App I have.
These are built using PYTHON including my App

The multiple Raga - some times there are 2, 3, 4 raga, RTP for slokham/virutham i can replace it with Ragamalika

Raman

Sath
Posts: 13
Joined: 10 Jan 2020, 12:37
x 1

#18 Re: Carnatic Youtube Collection

Post by Sath »

Great, thank you for the response.

Is it possible to have each row have a youtube link (in addition to the play button?) This'll be so much more helpful at least for me.

I mean: is the code that indexes and generates your website open source? If so, a link would be most appreciated.

Ragamalika is a good idea. However, it is valuable to know the ragas in it. One idea is to prepend 'Ragamalija' so all Ragamalikas are grouped together in the sorting. Eg:

"Dhanashree,Natabhairavi" would become:
"Ragamalika,Dhanashree,Natabhairavi"

There are probably even better ways. I'll try to think of some and post later.

Sath
Posts: 13
Joined: 10 Jan 2020, 12:37
x 1

#19 Re: Carnatic Youtube Collection

Post by Sath »

Any plans or chance in making the raw database available so one can query directly into it?

vsn69
Posts: 117
Joined: 02 Oct 2017, 17:30
x 23

#20 Re: Carnatic Youtube Collection

Post by vsn69 »

Amazing collection @rajanimalavi
Appreciate your work very, very much.

See if you can these as well
https://www.youtube.com/playlist?list=P ... zU6Idn-ygJ

She's Smt Rajam Pushpavanam, daughter of Madurai Pushpavanam and cousin of my Madurai Mani Iyer.
And my mother.
Thanks and best regards,

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#21 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Now, the carnatic youtbe video count stands around 66,500+
Fixed many errors, and removed lots of film videos sneaked in because of some of the artists sing film songs.
Also added transliteration for 6 languages: this needs more work, over time it will get fixed.

@vsn69 - i have added Smt Rajam Pushpavanam

Raman

vsn69
Posts: 117
Joined: 02 Oct 2017, 17:30
x 23

#22 Re: Carnatic Youtube Collection

Post by vsn69 »

ranjanimalavi wrote: 27 Aug 2020, 21:28 Now, the carnatic youtbe video count stands around 66,500+
Fixed many errors, and removed lots of film videos sneaked in because of some of the artists sing film songs.
Also added transliteration for 6 languages: this needs more work, over time it will get fixed.

@vsn69 - i have added Smt Rajam Pushpavanam

Raman
Truly great work you are doing.
And thank you.

sankark
Posts: 1993
Joined: 16 Dec 2008, 09:10
x 100
x 27

#23 Re: Carnatic Youtube Collection

Post by sankark »

Amazing & a valuable contribution.

Wondering: did you write an RPA BOT to scrape YT and build this index?

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#24 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

No, Crawled using pyppeteer a python package

Sath
Posts: 13
Joined: 10 Jan 2020, 12:37
x 1

#25 Re: Carnatic Youtube Collection

Post by Sath »

Any plans or chance in making the raw database available so one can query directly into it?

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#26 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

when i move from Git to a server, i will do that.
these are statically hosted pages

S.M. Vasudevan
Posts: 9
Joined: 20 Jun 2020, 06:48
x 2

#27 Re: Carnatic Youtube Collection

Post by S.M. Vasudevan »

Hello. I have posted a Madurai Mani Iyer rendition of Sarasa Sama Dana. It is from what I have seen the best quality rendition. I have made the Audio better and digitized it. It is very special with he elaboration and length. Here is the link: https://www.youtube.com/watch?v=Y_B_JZpUi1c&t=414s

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#28 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

Its more than 71K videos.
I have added a search box, which will work phonetically in most cases unless a word is broken.
Corrected many Tamil Lyrics as the rest of the languages are little easier.

Sath,
I converted all data to json, see if you can use it as it is.

Raman

shankarank
Posts: 3396
Joined: 15 Jun 2009, 07:16
x 4
x 18

#29 Re: Carnatic Youtube Collection

Post by shankarank »

And to process json here is a command line language.

https://stedolan.github.io/jq/download/

Everything is done on the fly, entire soup to nuts by chaining filters and aggregates.

Databases like postgre / mysql also support turning JSON content into a projected table, but the above is powerful , don't have to load it anywhere.

Sath
Posts: 13
Joined: 10 Jan 2020, 12:37
x 1

#30 Re: Carnatic Youtube Collection

Post by Sath »

Json would be fantastic. Raman: could you please point me to the json? I'm well versed in using it and so on, just need a url or other means to access. Thanks so much again, this is terrific!

ranjanimalavi
Posts: 276
Joined: 17 Feb 2007, 06:15
x 6

#31 Re: Carnatic Youtube Collection

Post by ranjanimalavi »

My Site: https://ramanarunachalam.github.io/Musi ... natic.html

Everything else is in github only: https://github.com/ramanarunachalam/Mus ... r/Carnatic

whoever wants can design their own website based on the json data

Raman

Post Reply