Setting up a new torrent site. Who is willing to help? (take 1) [DISCONTINUED]
-
- Posts: 133
- Joined: Feb 14th, '09, 04:05
Just a couple ideas but no computer knowledge to back it up... I noticed that as soon as the 'Jdrama of the month' choices are announced, people start seeding to back up both choices right away, I guess their hope is that when the winner is announced that there will be a larger pool of seeds available. Even after the time period expires there are always a number of people who keep seeding. So I wonder if it would be feasible to create a 'pledge seed' forum page where people would announce/pledge to seed a complete drama of their choice (raw or subbed) for the duration of one month, and name start and end dates. My assumption would be that even if that pledging person has to shut down their computer for hours each day, there would be some people who have quick download speeds and could actually get the whole thing within 1 day or so and keep it going. There would be no reason to worry about hit and run or anything, I've always seen good natured people on d-addicts who keep the ball rolling and seed for a good long while. This would make it easy for people to quickly look at the page and see what's currently pledged to be seeding. One filter I really liked on the torrent page was 'currently seeding' category, also the filter to select only jdramas or kdramas, etc. Even if the pledge dates' end came and went, people could look at the info and infer that someone out there might still be seeding and they could quickly pick up a whole drama that way. I think it would be a way to narrow down the number of downloads that stay half finished for eons, forlornly waiting for a seed.
Well, and I miss the shout box, it was always a quick way to find out what someone was looking for. Would love to see the shout box return.
Well, and I miss the shout box, it was always a quick way to find out what someone was looking for. Would love to see the shout box return.
Greetings all,
We are currently looking for Korean, Japanese, Chinese, Spanish and any other international language speakers to help us translate D-A v2.
Please contact us on http://d-addicts.tumblr.com if interested.
Cheers,
We are currently looking for Korean, Japanese, Chinese, Spanish and any other international language speakers to help us translate D-A v2.
Please contact us on http://d-addicts.tumblr.com if interested.
Cheers,
-
- Posts: 3
- Joined: Nov 7th, '06, 15:50
-
- Fansubber
- Posts: 36
- Joined: Oct 28th, '05, 20:18
- Location: between Takki and Eeteuk.
Can't wait to use the new site. I'm based in Korea so I shouldn't help out a the kdramas but I can help upload a bunch of random jtv that I have (mostly Johnny's based) and can get a lot of the recent things. I can also offer myself up as a mod or even a donator. I can also translate a bit of jp and kr. And I have fansub experience though it's been a while haha.
Translate subtitles to english? (From spanish) Or moderate the site in a spanish section?AyoYura wrote:Greetings all,
We are currently looking for Korean, Japanese, Chinese, Spanish and any other international language speakers to help us translate D-A v2.
Please contact us on http://d-addicts.tumblr.com if interested.
Cheers,
Hello, I have just joined today, but I have been using the site for a while..What a sad news!
I would like to help for a new site, but I have no knowledge whatsoever about torrents and the rest.... I have a bit of time though! If there could be anyway that I could learn how to do something useful for the new website please let me know!!!
I can translate French into English or English into French...
I would like to help for a new site, but I have no knowledge whatsoever about torrents and the rest.... I have a bit of time though! If there could be anyway that I could learn how to do something useful for the new website please let me know!!!
I can translate French into English or English into French...
AyoYura wrote:Greetings all,
We are currently looking for Korean, Japanese, Chinese, Spanish and any other international language speakers to help us translate D-A v2.
Please contact us on http://d-addicts.tumblr.com if interested.
Cheers,
I am fluent in English and Spanish. Also I understand quite some Japanese and Korean but not enough for a quality translation. You can send me a private message and I will give you my email. Thanks.
-
- Posts: 45
- Joined: Jan 11th, '11, 14:22
- Contact:
My 2c...
As a JPS user, I really like "gazelle" tracker software created by waht dot cd. If you don't feel like reinventing the wheel I would highly recommend using/customizing it.
http://whatcd.github.io/Gazelle/
Tracker: Don't have one. DHT only.
The old torrents: I have been checking a few old magnets (over 1 year old) from the old listing here; 90% are dead. Might as well abandon/can the old ones unless you can prove a seeder still exists.
Good luck with the new indexer!
As a JPS user, I really like "gazelle" tracker software created by waht dot cd. If you don't feel like reinventing the wheel I would highly recommend using/customizing it.
http://whatcd.github.io/Gazelle/
Tracker: Don't have one. DHT only.
The old torrents: I have been checking a few old magnets (over 1 year old) from the old listing here; 90% are dead. Might as well abandon/can the old ones unless you can prove a seeder still exists.
Good luck with the new indexer!
Some issues for uploaders with a torrent page like the coalgirls one (especially older shows):
- non-standard resolution (i.e. not 480p/720p/1080p, etc.)
- unknown source (i.e. don't know if it's tv or dvd rip)
- I find it helpful to know which group encoded a file. Rather than the Series category, I'd rather just see the file/folder name.
Would there be a way to protect users from being identified if there was a shutdown? I'd be more comfortable uploading again if I knew there wasn't any incriminating info about me.
Re: allowing non-English hardsubs/softsubs. I know a few subbers originally subbed in another language and then moved to English subbing - making it harder for these subbers to work will be a detriment to the community. Maybe we could have two categories - English subbed and Non-English subbed.
How about a RAW batch upload team? So when a series is finished by an encoding team, a batch torrent of that series is put up and individual episode torrents removed. Some uploaders up their own batch torrents, so this team wouldn't upload a torrent until let's say 2 weeks after the last individual torrent was uploaded. Batch torrents stay alive much longer than individual torrents and it's annoying reseeding a full series and having to download 10 different torrent files.
Please take your time with the new site. Sorry I can't be of much help besides offering suggestions and uploading.
- non-standard resolution (i.e. not 480p/720p/1080p, etc.)
- unknown source (i.e. don't know if it's tv or dvd rip)
- I find it helpful to know which group encoded a file. Rather than the Series category, I'd rather just see the file/folder name.
Would there be a way to protect users from being identified if there was a shutdown? I'd be more comfortable uploading again if I knew there wasn't any incriminating info about me.
Re: allowing non-English hardsubs/softsubs. I know a few subbers originally subbed in another language and then moved to English subbing - making it harder for these subbers to work will be a detriment to the community. Maybe we could have two categories - English subbed and Non-English subbed.
How about a RAW batch upload team? So when a series is finished by an encoding team, a batch torrent of that series is put up and individual episode torrents removed. Some uploaders up their own batch torrents, so this team wouldn't upload a torrent until let's say 2 weeks after the last individual torrent was uploaded. Batch torrents stay alive much longer than individual torrents and it's annoying reseeding a full series and having to download 10 different torrent files.
Please take your time with the new site. Sorry I can't be of much help besides offering suggestions and uploading.
-
- Posts: 1
- Joined: Nov 30th, '14, 23:05
most of the streaming sites hard sub their shows anyway; AT has the subs separately, and they can be downloaded without effecting ratio, or getting the torrent there.hey9 wrote:Some issues for uploaders with a torrent page like the coalgirls one (especially older shows):
- non-standard resolution (i.e. not 480p/720p/1080p, etc.)
- unknown source (i.e. don't know if it's tv or dvd rip)
- I find it helpful to know which group encoded a file. Rather than the Series category, I'd rather just see the file/folder name.
Would there be a way to protect users from being identified if there was a shutdown? I'd be more comfortable uploading again if I knew there wasn't any incriminating info about me.
Re: allowing non-English hardsubs/softsubs. I know a few subbers originally subbed in another language and then moved to English subbing - making it harder for these subbers to work will be a detriment to the community. Maybe we could have two categories - English subbed and Non-English subbed.
How about a RAW batch upload team? So when a series is finished by an encoding team, a batch torrent of that series is put up and individual episode torrents removed. Some uploaders up their own batch torrents, so this team wouldn't upload a torrent until let's say 2 weeks after the last individual torrent was uploaded. Batch torrents stay alive much longer than individual torrents and it's annoying reseeding a full series and having to download 10 different torrent files.
Please take your time with the new site. Sorry I can't be of much help besides offering suggestions and uploading.
regarding torrent batches, it does not matter if there is a real torrent (not just magnet link) as one can download the eps one needs
I agree with the first feedback over there. Here's a sample of a torrent site that is convenient without restrictins: http://yts.re/browse-movieAyoYura wrote:For those who would like to check on the site's progress can do so here http://d-addicts.tumblr.com
We'll be updating the blog every time progress is made.
Regards,
****
Quick Teaser on what is to come https://coalgirls.wakku.to/torrent-listing
Sorry I wasn't clear, my comment was only in regards to hardsubs, and was in response to feedback left on the d-a 2.0 tumblr suggesting only English hardsubs - and not other language hardsubs - be allowed on the tracker/index.cindy25 wrote:most of the streaming sites hard sub their shows anyway; AT has the subs separately, and they can be downloaded without effecting ratio, or getting the torrent there.hey9 wrote:Some issues for uploaders with a torrent page like the coalgirls one (especially older shows):
- non-standard resolution (i.e. not 480p/720p/1080p, etc.)
- unknown source (i.e. don't know if it's tv or dvd rip)
- I find it helpful to know which group encoded a file. Rather than the Series category, I'd rather just see the file/folder name.
Would there be a way to protect users from being identified if there was a shutdown? I'd be more comfortable uploading again if I knew there wasn't any incriminating info about me.
Re: allowing non-English hardsubs/softsubs. I know a few subbers originally subbed in another language and then moved to English subbing - making it harder for these subbers to work will be a detriment to the community. Maybe we could have two categories - English subbed and Non-English subbed.
How about a RAW batch upload team? So when a series is finished by an encoding team, a batch torrent of that series is put up and individual episode torrents removed. Some uploaders up their own batch torrents, so this team wouldn't upload a torrent until let's say 2 weeks after the last individual torrent was uploaded. Batch torrents stay alive much longer than individual torrents and it's annoying reseeding a full series and having to download 10 different torrent files.
Please take your time with the new site. Sorry I can't be of much help besides offering suggestions and uploading.
regarding torrent batches, it does not matter if there is a real torrent (not just magnet link) as one can download the eps one needs
I'm not sure what you mean about the torrent batches.
-
- Posts: 2
- Joined: Nov 24th, '14, 02:35
I agree. Batch torrent for finished series are more convinient for downloaders. Besides, if someone don't want to download all files, they can make a selection after they finished download the torrent file. Batch torrent can guarantee downloaders to get a complete series.hey9 wrote:How about a RAW batch upload team? So when a series is finished by an encoding team, a batch torrent of that series is put up and individual episode torrents removed. Some uploaders up their own batch torrents, so this team wouldn't upload a torrent until let's say 2 weeks after the last individual torrent was uploaded. Batch torrents stay alive much longer than individual torrents and it's annoying reseeding a full series and having to download 10 different torrent files..
yes. i've downloaded some pre-batch files and halfway thru the season, the titling format changed. so for a new site would it be better to standardize the names so they follow each other or keep the originals which might not match up?jlo8 wrote:I do have quite a bit of dramas in my hdd over the years as well. However, there are some dramas whereby I've changed the file names that are different from the initial uploads.
I have both hardsubs and softsubs files, so it's gonna get a bit complicated as well.
This site is literally the only source of my home entertainment for the past decade or so, was totally devastated when I came home from vacation to find the torrent section dead.
As with many others here, will be willing to seed stuff that I have accumulated over the years, make some donations within my means.
Please start this.
As with many others here, will be willing to seed stuff that I have accumulated over the years, make some donations within my means.
Please start this.
-
- Posts: 12
- Joined: Dec 27th, '05, 18:20
As many have said before me. I have little in the way of technical skill but I have a bunch of Dramas and I am willing to help with uploading. DA has been my life for quite some time now, and I have such fond memories of finding new actors/actresses and coming across awesome shows... So anything I can help with I would like to.
Also, I know it is niche, but I have a medical background and basic Japanese skills, so if any group needs a terminology checker for medical shows, EMS or Fire shows. I can probably help with that.
Also, I know it is niche, but I have a medical background and basic Japanese skills, so if any group needs a terminology checker for medical shows, EMS or Fire shows. I can probably help with that.
-
- Posts: 96
- Joined: Nov 21st, '12, 04:12
- Location: SG
-
- Posts: 80
- Joined: Jun 3rd, '04, 07:12
- Location: Piteå
Why not extract the subs yourself and do the edit with Aegisub?HimaOtaku wrote:I'd like to see some better fansubs for old series actually. I downloaded a batch torrent of Dai Sensei Onizuka (GTO) and I felt my head bursts into flames as I watched it with the subs included. Sub files nowadays should all be .ass.
Because I would need a translator to check that the sentences are actually what is being said. The problem with the subs was horrendous grammar and infinite typos. That's why I think it'd be nice if there's a fansub group dedicated to redoing old series that didn't get the .ass love.karyuudo wrote:Why not extract the subs yourself and do the edit with Aegisub?HimaOtaku wrote:I'd like to see some better fansubs for old series actually. I downloaded a batch torrent of Dai Sensei Onizuka (GTO) and I felt my head bursts into flames as I watched it with the subs included. Sub files nowadays should all be .ass.
Well I do fansubbing myself, but I'm tied up on a 52 series at the moment that I want to get done before mid Dec.HimaOtaku wrote:Because I would need a translator to check that the sentences are actually what is being said. The problem with the subs was horrendous grammar and infinite typos. That's why I think it'd be nice if there's a fansub group dedicated to redoing old series that didn't get the .ass love.karyuudo wrote:Why not extract the subs yourself and do the edit with Aegisub?HimaOtaku wrote:I'd like to see some better fansubs for old series actually. I downloaded a batch torrent of Dai Sensei Onizuka (GTO) and I felt my head bursts into flames as I watched it with the subs included. Sub files nowadays should all be .ass.
I have a question - there are quite a few WOWOW SPs that were originally aired on tv but eventually released in theatres. Would these count as TV specials (allowed) or movies (not allowed)?
I always rename my files too, it is a pain to change the names back just to reseed.
If I got it from d-addicts, I will keep the original name. If I didn't and the file name is in Japanese, I will change it to the English name. Stripping the file of encoder tags (e.g. DoA, YYeTs) is a no-no. I think it's best to rename to the original file if possible - that way people can quickly be able to tell whether they have the same file as you or not and can help seed it.tomzero wrote:yes. i've downloaded some pre-batch files and halfway thru the season, the titling format changed. so for a new site would it be better to standardize the names so they follow each other or keep the originals which might not match up?jlo8 wrote:I do have quite a bit of dramas in my hdd over the years as well. However, there are some dramas whereby I've changed the file names that are different from the initial uploads.
I have both hardsubs and softsubs files, so it's gonna get a bit complicated as well.
I always rename my files too, it is a pain to change the names back just to reseed.
What is everyone trending towards? Nyaa, Asia Torrents or a new startup site?
For those who need drama now, there are host based sites offering drama and TV. One is: http://jpfree365.com
You can search for other sites by goggling the drama name followed by a file hosters name.
Examples:
ドクターX 3 rapidgator
ザ!鉄腕!DASH!! uploadable
Doctor x 3 Ryushare
Tetsuwan Dash Extmatrix
For those who need drama now, there are host based sites offering drama and TV. One is: http://jpfree365.com
You can search for other sites by goggling the drama name followed by a file hosters name.
Examples:
ドクターX 3 rapidgator
ザ!鉄腕!DASH!! uploadable
Doctor x 3 Ryushare
Tetsuwan Dash Extmatrix
- Keiko1981
- Administrator
- Posts: 7574
- Joined: Apr 9th, '06, 11:27
- Location: Sweden
- Has thanked: 5 times
- Been thanked: 82 times
- Contact:
To me it looks like Nyaa is being used for now, until the new site is up and running.chiuchimu wrote:What is everyone trending towards? Nyaa, Asia Torrents or a new startup site?
For those who need drama now, there are host based sites offering drama and TV. One is: http://jpfree365.com
You can search for other sites by goggling the drama name followed by a file hosters name.
Examples:
ドクターX 3 rapidgator
ザ!鉄腕!DASH!! uploadable
Doctor x 3 Ryushare
Tetsuwan Dash Extmatrix
See also these: http://www.giftedones.net/links-of-interest/
Or try to use Bing instead of Google. You may end up with more/different results.
It's a pain, but I just want to make sure any potential contributors realize that it's the ONLY way to re-seed. Changing the name will change the infohash (unlike changing the trackers or comments) and you will either be the ONLY seeder or nobody will trust or bother to download your torrent.hey9 wrote:I have a question - there are quite a few WOWOW SPs that were originally aired on tv but eventually released in theatres. Would these count as TV specials (allowed) or movies (not allowed)?
If I got it from d-addicts, I will keep the original name. If I didn't and the file name is in Japanese, I will change it to the English name. Stripping the file of encoder tags (e.g. DoA, YYeTs) is a no-no. I think it's best to rename to the original file if possible - that way people can quickly be able to tell whether they have the same file as you or not and can help seed it.tomzero wrote:yes. i've downloaded some pre-batch files and halfway thru the season, the titling format changed. so for a new site would it be better to standardize the names so they follow each other or keep the originals which might not match up?jlo8 wrote:I do have quite a bit of dramas in my hdd over the years as well. However, there are some dramas whereby I've changed the file names that are different from the initial uploads.
I have both hardsubs and softsubs files, so it's gonna get a bit complicated as well.
I always rename my files too, it is a pain to change the names back just to reseed.
That's why it was important to recover the old D-A torrents (or at least their infohashes). Someone got most of them and posted the list here, but not the latest (since November), and someone else got all the rest, but didn't post them here. Well, those are new enough that they can probably be found again, if we don't wait too long - but I haven't seen any consensus on doing this. If you needed to, I think I could help (there are only about 3 missing weeks).
A non-D-A uploader on Nyaa posted re-named, new torrents to Live Action Raw and attracted no seeders (see the Birth of a Beauty E10 WITH with no seeders and 6 comments).
As far as our regular postings on Nyaa are concerned, there have been a few collisions... why don't we go back to organizing the uploads here again - if that continues? So far, it's been working really well, but nothing's been formalized, so let's hope it otherwise continues as smoothly as it has been this past week.
Another point: A lot of D-A uploaders are posting on Nyaa as "Anonymous". But if we like to "trust" our D-A uploaders, shouldn't we login to post? Notice that if you login to post, users who want to follow your work can click on your name to sublist all your titles they are following (and the site will appear less "disorganized," as some have complained).
I have already done that with VampireXxX postings and have discovered some new JDramas that I wouldn't have found otherwise. VampireXxX is also the only one tagging the uploads with [JDrama] which is really helpful. We should probably standardize on that, but it has been working well either way.
chanpuru wrote:
There's a potential mess over on Nyaa, though. Today someone added the exact same files I've uploaded of Akka, which just makes me miss the organization among uploaders we had here.
I'm only uploading one drama over on Nyaa ("Akka" from WOWOW, 720p version), but I also add [Jdrama] before the title, taking VampireXxX's example to distinguish.VampireXxX is also the only one tagging the uploads with [JDrama] which is really helpful.
There's a potential mess over on Nyaa, though. Today someone added the exact same files I've uploaded of Akka, which just makes me miss the organization among uploaders we had here.
Yep, there might be a ton of people with old d-a files but no idea how to reseed. The process should be as easy as possible. I rename my files to reseed and rename them back to the way I like when I'm finished. This applies for re-uploads too.chanpuru wrote:It's a pain, but I just want to make sure any potential contributors realize that it's the ONLY way to re-seed. Changing the name will change the infohash (unlike changing the trackers or comments) and you will either be the ONLY seeder or nobody will trust or bother to download your torrent.
That's why it was important to recover the old D-A torrents (or at least their infohashes). Someone got most of them and posted the list here, but not the latest (since November), and someone else got all the rest, but didn't post them here. Well, those are new enough that they can probably be found again, if we don't wait too long - but I haven't seen any consensus on doing this. If you needed to, I think I could help (there are only about 3 missing weeks).
A non-D-A uploader on Nyaa posted re-named, new torrents to Live Action Raw and attracted no seeders (see the Birth of a Beauty E10 WITH with no seeders and 6 comments).
As far as our regular postings on Nyaa are concerned, there have been a few collisions... why don't we go back to organizing the uploads here again - if that continues? So far, it's been working really well, but nothing's been formalized, so let's hope it otherwise continues as smoothly as it has been this past week.
Another point: A lot of D-A uploaders are posting on Nyaa as "Anonymous". But if we like to "trust" our D-A uploaders, shouldn't we login to post? Notice that if you login to post, users who want to follow your work can click on your name to sublist all your titles they are following (and the site will appear less "disorganized," as some have complained).
I have already done that with VampireXxX postings and have discovered some new JDramas that I wouldn't have found otherwise. VampireXxX is also the only one tagging the uploads with [JDrama] which is really helpful. We should probably standardize on that, but it has been working well either way.
We should have some consensus on file names. I'd at least like to know what to do with files with Asian characters in the file name. Some people don't have them installed on their PC. English speakers are naturally gonna rename files to English, and majority probably won't be bothered to rename them back if the torrent needs to be reseeded..
People might be Anonymous because they're nervous after the da shutdown. For now I'm at AT until da2. Tbh if da2 has the anonymous option I'll probably use it.
-
- Posts: 96
- Joined: Nov 21st, '12, 04:12
- Location: SG
Uh, that was me. I was working on editing the files to leave just the torrent info and links but I got discouraged. I believe I did finish the missing 21 pages so I'll post those a little later.chanpuru wrote:That's why it was important to recover the old D-A torrents (or at least their infohashes). Someone got most of them and posted the list here, but not the latest (since November), and someone else got all the rest, but didn't post them here. Well, those are new enough that they can probably be found again, if we don't wait too long - but I haven't seen any consensus on doing this. If you needed to, I think I could help (there are only about 3 missing weeks).
Edit: Last 21 pages of torrents posted to D-addicts before the torrents section was closed. To fill the gap in xaxa's list.
https://www.sendspace.com/file/nczq9h
chanpuru, saigo-x,
I'm still in the process of downloading all the torrents to all the hashes I have in my database (51272), in that process resetting the 6-month timeout zoink and others have imposed; after that I'll scan the rescue-files that were posted here, to check what was missing in my database
...this will still take some time...
By then I'd hoped DA2 would be up and running...and I could provide them with the data.
Or we will work on a different solution.
nothing is lost yet or only just a little, the only concern I have is that this (tiny) band of friends will grow impatient and leave...
I'm still in the process of downloading all the torrents to all the hashes I have in my database (51272), in that process resetting the 6-month timeout zoink and others have imposed; after that I'll scan the rescue-files that were posted here, to check what was missing in my database
...this will still take some time...
By then I'd hoped DA2 would be up and running...and I could provide them with the data.
Or we will work on a different solution.
nothing is lost yet or only just a little, the only concern I have is that this (tiny) band of friends will grow impatient and leave...
- Pavlova Megaflop
- Posts: 70
- Joined: Mar 25th, '09, 07:46
- Has thanked: 3 times
- Been thanked: 2 times
Would you be able to give (or link to) simple instructions as to how to use the archive?godane wrote:...I helped saved d-addicts files over a year ago. I'm also part of the archiveteam. We have a archivebot that saved most of the recent stuff i think...
How can we match the list of zoink hashes with the doramas they represent?
Thanks!
Awesome! I was able to download all but 25 or so of the 1500 torrent files for the last/newest 600 pages of the torrent section before it was closed. I have already located about half of the missing torrents files on other indexing sites. Let me know if they could be of any help to you.koshonin wrote:I'm still in the process of downloading all the torrents to all the hashes I have in my database (51272), in that process resetting the 6-month timeout zoink and others have imposed; after that I'll scan the rescue-files that were posted here, to check what was missing in my database
...this will still take some time...
Thank you!, lets see how far I getsaigo_x wrote:Let me know if they could be of any help to you.
Is there a way to extract the original torrent site layoutgodane wrote:Hey everyone
This is my first time posting. I helped saved d-addicts files over a year ago. I'm also part of the archiveteam. We have a archivebot that saved most of the recent stuff i think.
viewtopic urls: https://archive.org/details/www.d-addic ... c-20131112
viewforum urls: https://archive.org/details/www.d-addic ... m-20131111
zoink.it torrent urls: https://archive.org/details/www.d-addic ... t-20131115
download/subfiles urls: https://archive.org/details/www.d-addic ... 5-20131113
This i hope is very help full with setting up your new site.
like [dramatype--sub--title--size--links]
from those archive-files? That would be awesome.
And also: which program is used for a meaningful handling of warc-files? I skimmed the documentation but didn't find any.
---
To those, who upload @ nyaa: I'm going to scan those with usernames like VampireXxX, Paz (as they have a usable rss-feed associated to their name) to integrate them in the resulting db
If you object to it, then please PM me and I'll leave you out.
To all the one doing their best and taking time to retrieve all the data and all technical stuff I don't understand and especially Koshonin,
I think that we are all grateful for your work and I don't think people will grow impatient and leave... They are fans and they know you do it in your spare time for free!!!!!!! but if they do then it would not be a great loss, people who uses others and cannot appreciate what is done are just parasites!!!!
Thank you for everything you did/ do/ will do to allow us enjoy Asian dramas and the rest !
I think that we are all grateful for your work and I don't think people will grow impatient and leave... They are fans and they know you do it in your spare time for free!!!!!!! but if they do then it would not be a great loss, people who uses others and cannot appreciate what is done are just parasites!!!!
Thank you for everything you did/ do/ will do to allow us enjoy Asian dramas and the rest !
Just found this somewhere at d-a (I forgot), and this is still alive I guess..koshonin wrote: Is there a way to extract the original torrent site layout
like [dramatype--sub--title--size--links]
from those archive-files? That would be awesome.
And also: which program is used for a meaningful handling of warc-files? I skimmed the documentation but didn't find any.
here
Thank you!zenith777 wrote:Just found this somewhere at d-a (I forgot), and this is still alive I guess..
here
it's from xaxa: http://www.d-addicts.com/forum/viewtopi ... tm#1705893
I already added them which accounted for ~3k additional entries.
I just managed to scan one of those archive.org files, that would add another 55k hashes to the 51k I already have (probably including all the k-c-hk-tw-sg files I didn't care for when I set up my db)
but here is the question: I have no meaningful way to categorize them by type(jdrama..), sub nor date**; resulting only in the basic data like: hash, size*** and name***, but the name is more often than not in the original language and not in the transcribed/translated form used to identify the torrent here in the postings which made it easily searchable...
...even if I were to make an alphabetic sorted catalog from them I doubt that it would be of much use to the majority.
So... what do you think?
what are the plans of the DA2 team?
[** (maybe I can extract the date from the torrent, but that is not always possible)]
[*** would need to download the torrentfile first to get them, so for 55k and avoiding flooding, will probably take quite some time]
<s>You know I finally realized something which makes things alot easier. The Internet Archive Wayback Machine has a backup of D-Addicts from July 4th 2014. We can use that to recover 2947 of the 3473 pages in the torrent section. We can then use my backup of the last/newer 600 pages to recover the missing 526. Since the pages are all structured identically all we need is a script to be able to extract the torrent info and links from each page. I was trying to do just that before the torrent section was closed, but I lack the knowledge and skill to do it. Any volunteers?koshonin wrote:I just managed to scan one of those archive.org files, that would add another 55k hashes to the 51k I already have (probably including all the k-c-hk-tw-sg files I didn't care for when I set up my db)
but here is the question: I have no meaningful way to categorize them by type(jdrama..), sub nor date**; resulting only in the basic data like: hash, size*** and name***, but the name is more often than not in the original language and not in the transcribed/translated form used to identify the torrent here in the postings which made it easily searchable...
...even if I were to make an alphabetic sorted catalog from them I doubt that it would be of much use to the majority.
Edit: I shoudl say that everyone can access a working backup of the D-Addicts torrent section that contains everything from Jully 3 2014 and older.
http://web.archive.org/web/201407031642 ... rrents.php</s>
<sigh> Nevrmind...it only saved the last 23 pages...
Unfortunately it looks like the torrent pages are not complete. The newest record only saved 23 pages. I've browsed through other record dates and they also only have a small number of the torrent pages. It looks like the torrent comment threads are a little more complete, but have the same problem. The Wayback strucutre is not very good for traversing different records. I can't phatom writing a script to accurately traverse the records.koshonin wrote:saigo_x,
that is actually a good find, I will try to make a script for it
looks like little by little we are getting somewhere
I wish godane would reply so we can find out if the archive he linked to is similarly incomplete. I did a little research and the only way to open up the archive is to load a Wayback Machine variant on a Nix system using Java and Tomcat (guide).
I think I have most of the newer ones and the pages you were able to gather will also help, I haven't run a script over them yet.saigo_x wrote:Unfortunately it looks like the torrent pages are not complete. The newest record only saved 23 pages. I've browsed through other record dates and they also only have a small number of the torrent pages.
It looks like the torrent comment threads are a little more complete, but have the same problem. The Wayback strucutre is not very good for traversing different records. I can't phatom writing a script to accurately traverse the records.
I was more concerned with the 55k additional hashes I found in the archive.org-zoink file, but thanks to you discovering the wayback link I think we may be able to stitch a lot of information together (if it's not in one snapshot than it might be in another, as long as the magnet/hash-identifying link is in the table. The really old archives have the format 'filename.torrent', so they wont be of any help)
on the point of traversing:
that should be fairly easy; replacing the XXXX in
[http://web.archive.org/web/201407031642 ... start=XXXX]
by 0,25,50,..., 80900
that's how I will try it
Another good find, at least for reference (I don't want to install java)saigo_x wrote:I wish godane would reply so we can find out if the archive he linked to is similarly incomplete. I did a little research and the only way to open up the archive is to load a Wayback Machine variant on a Nix system using Java and Tomcat (guide).
Unfortunately that doesn't work becasue it grabs pages from different records/dates. Open up the 20140703164231 record and go into the Torrent Comments. If you click page 2 you will get a page from 20140501201825. Not only are the pages random, but some are missing as well.koshonin wrote:on the point of traversing:
that should be fairly easy; replacing the XXXX in
[http://web.archive.org/web/201407031642 ... start=XXXX]
by 0,25,50,..., 80900
that's how I will try it
It looks like Heritrix will run on Windows but I am still looking for a guide.koshonin wrote:Another good find, at least for reference (I don't want to install java)
you're right, I see it now. arrgh !!saigo_x wrote:Unfortunately that doesn't work becasue it grabs pages from different records/dates. Open up the 20140703164231 record and go into the Torrent Comments. If you click page 2 you will get a page from 20140501201825. Not only are the pages random, but some are missing as well.
I think Heritrix is the spider used by archive org (also running on java)saigo_x wrote:It looks like Heritrix will run on Windows but I am still looking for a guide.
just thinking about it: someone did ask the admin already if they could get a copy of the database table, right? just the[ type-title-uploader-date-link/hash-size] bit. and he/she said no, right?
Looks like Heritrix is only for creating the archives. I am goign to try WAIL. Doesn't really appear to be many WARC viewers.koshonin wrote:I think Heritrix is the spider used by archive org (also running on java)saigo_x wrote:It looks like Heritrix will run on Windows but I am still looking for a guide.
Yeah, I asked a number of times but never got a response. We were lucky Rusoshin brought back the torrent comments pages even if they are locked.koshonin wrote:just thinking about it: someone did ask the admin already if they could get a copy of the database table, right? and he/she said no, right?
If you want the torrent database then just ask Ruroshin.
There is no need to scrape data from google cache or any internet archivers.
We still have all the data (86912 torrents).
Ruro just removed the db access (through torrents.php) for some reason which I don't know either.
this is our table structure (obviously with tracker stats from the old days before 2010):
all you may want is:
* info_hash (hash from torrent info)
* filename (file or folder name from torrent info)
* info (size in MB from torrent info)
* name (alternative display name if given by uploader, otherwise same as filename, could be edited by uploader or mods)
* type (jdrama/kdrama/j-tv/ost/.. etc.)
* date_added ((re)upload timestamp)
* sub (RAW/english/etc.)
types in database:
sub types in database:
ask Ruroshin kindly for a sql-dump of the columns you are interested in.
there is even a second table (archive) which contains all the "deleted" torrents.
--MoerkJ
There is no need to scrape data from google cache or any internet archivers.
We still have all the data (86912 torrents).
Ruro just removed the db access (through torrents.php) for some reason which I don't know either.
Code: Select all
$> "SELECT count(*) FROM torrents"
+----------+
| count(*) |
+----------+
| 86912 |
+----------+
Code: Select all
$> "DESCRIBE torrents"
+---------------+--------------+------+-----+---------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+---------------------+-------+
| info_hash | varchar(40) | NO | PRI | | |
| filename | varchar(250) | YES | | NULL | |
| url | varchar(250) | NO | | | |
| info | varchar(250) | NO | | | |
| name | varchar(100) | YES | MUL | NULL | |
| uploadBy | varchar(25) | YES | MUL | NULL | |
| type | varchar(100) | YES | MUL | NULL | |
| date_added | datetime | NO | | 0000-00-00 00:00:00 | |
| announce | varchar(255) | YES | | NULL | |
| seeds | int(11) | YES | | 0 | |
| leechers | int(11) | YES | | 0 | |
| finished | int(11) | YES | | 0 | |
| dl_link | varchar(255) | YES | | NULL | |
| group | varchar(250) | YES | | NULL | |
| keep | char(1) | YES | | N | |
| sub | varchar(10) | YES | MUL | unknown | |
| info_hash_hex | varchar(100) | YES | | NULL | |
+---------------+--------------+------+-----+---------------------+-------+
* info_hash (hash from torrent info)
* filename (file or folder name from torrent info)
* info (size in MB from torrent info)
* name (alternative display name if given by uploader, otherwise same as filename, could be edited by uploader or mods)
* type (jdrama/kdrama/j-tv/ost/.. etc.)
* date_added ((re)upload timestamp)
* sub (RAW/english/etc.)
types in database:
Code: Select all
$> "SELECT type, count(type) FROM torrents GROUP BY type"
+-------------+-------------+
| type | count(type) |
+-------------+-------------+
| | 4 |
| cdrama | 675 |
| hk-tv | 7 |
| hkdrama | 211 |
| j-tv | 13299 |
| jdrama | 27289 |
| k-tv | 5298 |
| kdrama | 37002 |
| ost | 529 |
| other | 73 |
| sgdrama | 71 |
| theme songs | 425 |
| twdrama | 2029 |
+-------------+-------------+
Code: Select all
"SELECT sub, count(sub) FROM torrents GROUP BY sub"
+----------+------------+
| sub | count(sub) |
+----------+------------+
| | 806 |
| Autres | 1 |
| English | 7492 |
| Japanese | 2314 |
| Korean | 93 |
| N/A | 1410 |
| Other | 5274 |
| RAW | 67816 |
| Simp Chn | 768 |
| Trad Chn | 880 |
| unknown | 58 |
+----------+------------+
there is even a second table (archive) which contains all the "deleted" torrents.
Code: Select all
$> "SELECT count(*) FROM torrents_archive"
+----------+
| count(*) |
+----------+
| 9137 |
+----------+
Ohh man you're great!MoerkJ wrote: If you want the torrent database then just ask Ruroshin.
There is no need to scrape data from google cache or any internet archivers.
We still have all the data (86912 torrents).
Ruro just removed the db access (through torrents.php) for some reason which I don't know either.
all you may want is:
* info_hash (hash from torrent info)
* filename (file or folder name from torrent info)
* info (size in MB from torrent info)
* name (alternative display name if given by uploader, otherwise same as filename, could be edited by uploader or mods)
* type (jdrama/kdrama/j-tv/ost/.. etc.)
* date_added ((re)upload timestamp)
* sub (RAW/english/etc.)
--MoerkJ
Yesterday, I try to download, understand and re-organize the torrent archieve/backups using ruby scripts and List.js (search engine). Even I try using beconder to extract the torrent info, but got a lot of bugs (escape literal and so on).
I knew you're one of this site's main programmer so maybe you can guide DA2 team in the internal meeting (PM Taido, if you don't mind )
If we can get the mysql dump data from Admins, that would save the daykoshonin wrote:So... what do you think?
what are the plans of the DA2 team?
I'm not sure if I can leak out the internal's meetings information, but the hint is DA2 will be focusing on torrents kinda like this, but keep the simplicity and ease of use.
Maybe you can help to give out some ideas in the tumblr shoutbox
I did try asking Ruroshin if we could get a backup/archive of the torrents but I never got a response. The suddenness of the closing and the fact that even you don't know why is concerning. Your help is appreciated though. I hope the guys from DA2 have better luck.MoerkJ wrote: If you want the torrent database then just ask Ruroshin.
There is no need to scrape data from google cache or any internet archivers.
We still have all the data (86912 torrents).
Ruro just removed the db access (through torrents.php) for some reason which I don't know either.
@ MoerkJ,
that was exactly what I was talking about
thank you!
@ zenith777,
I was just asking for some DA2 feedback concerning the extra 55k torrents, but all seems good now.
Good luck!
@ saigo_x,
so, what are we going to do with all the extra time now
'guess we watch some comedy, we had enough drama
that was exactly what I was talking about
thank you!
@ zenith777,
I was just asking for some DA2 feedback concerning the extra 55k torrents, but all seems good now.
Good luck!
@ saigo_x,
so, what are we going to do with all the extra time now
'guess we watch some comedy, we had enough drama
I'm here now. It looks like most of the torrent pages are saved.
http://web.archive.org/web/$star/http:/ ... tart=$star
NOTE: $star is Shift+8 character. I can' use that character otherwise nothing in this post will be displayed.
That make finding all the pages easier. Just know some pages got error pages but also got the right page the same day.
Also there is warc-proxy that alard made a few years back. It may make it easier to look at the web archive in firefox.
https://github.com/alard/warc-proxy
I hope this helps.
http://web.archive.org/web/$star/http:/ ... tart=$star
NOTE: $star is Shift+8 character. I can' use that character otherwise nothing in this post will be displayed.
That make finding all the pages easier. Just know some pages got error pages but also got the right page the same day.
Also there is warc-proxy that alard made a few years back. It may make it easier to look at the web archive in firefox.
https://github.com/alard/warc-proxy
I hope this helps.
Thanks for replying. That information is really useful especially the record list for the torrents page. However I'm just going to wait and see if the DA2 guys can get a copy of the database.godane wrote:I'm here now. It looks like most of the torrent pages are saved.
http://web.archive.org/web/$star/http:/ ... tart=$star
NOTE: $star is Shift+8 character. I can' use that character otherwise nothing in this post will be displayed.
That make finding all the pages easier. Just know some pages got error pages but also got the right page the same day.
Also there is warc-proxy that alard made a few years back. It may make it easier to look at the web archive in firefox.
https://github.com/alard/warc-proxy
I hope this helps.
I have quite a bit of pre-2011 lonha I can upload. It's on an external drive right now, so I don't have a complete overview of how good it is. I'm actually behind on 2011-onwards as I was in Japan for a year and just watched stuff live for a while. After I left, I really haven't had a chance to catch up, so if you (or anyone else) uploads 2012-2014 (in good quality), I'd really appreciate it.uedajirou wrote:I think my Lonha 2012-2014 is pretty complete (maybe >80% of them),tenkal wrote:I only have mostly youku 622x1024 [MQ] rips for around 2012-2013 and most of pre-2012 episodes are in 360-480px[LQ].. I'll make a list of my owarai collection public and only upload the ones in demand, I'm rather hesitant to share the LQ-MQ ones. Btw have you got any HD for LonHa pre-2013 preferably, PM me if you do.uedajirou wrote: Would you like to help uploading older (as well as new episode) of variety shows like London hearts in the new site?
Thanks
but for 2011, I only have september 2011 onwards.
Since I obtained them from d-addicts, there are variation of quality,
but I am sure most of my collections also MQ/LQ.
I hope you will share Lonha episode pre-2012 and also the upcoming 2014 episodes.
Thanks a lot:wub:
(Dont you mind sharing sources where you obtain Lonha episodes thru PM? Thanks
. I have downloaded perfect Dark, but i am confused as how to use it)
To be honest, 360p is enough for me to enjoy owarai shows, so yeah...
I also have a pretty complete collection of Hexagon 2, really sad they took the show off the air. Shinsuke had yakuza ties apparently, and the show died shortly after he was removed as host.
Forget Perfect Dark by the way - it sucks for owarai.
As for the new site, I would seriously love it if there was some kind of enforced naming standard - right now my J-TV section looks like a complete mess, and without proper naming it's a real pain in the ass to get things working and looking right in Plex.
Recovery-Project:
... I couldn't leave it alone
but with zoink down not much I could do.
so while still far from being complete,
here is some fallout from me hoovering over the data ,
then scraping some trackers(pbt)
and stitching the html together
a table of seeds=0 peers>1
it's ugly but maybe helpful
edit: link removed, due to lack of interest
... I couldn't leave it alone
but with zoink down not much I could do.
so while still far from being complete,
here is some fallout from me hoovering over the data ,
then scraping some trackers(pbt)
and stitching the html together
a table of seeds=0 peers>1
it's ugly but maybe helpful
edit: link removed, due to lack of interest
Last edited by koshonin on Dec 17th, '14, 03:50, edited 1 time in total.
Who is online
Users browsing this forum: No registered users and 0 guests