Migrate Large Bazaar Repositories to Git

20. March 2019 16:18 by Cameron in Source Control  //  Tags: , , ,   //   Comments

I recently was tasked with migrating my company's Bazaar repository to Git, but using Bazaar and Git for Windows wasn't working quite right. Every time I would try to do the migration, Bazaar would run out of memory. After further research, I found that this was due to Bazaar on Windows attempting to load all files into memory and this was exceeding the 32bit process memory limit of 2GB. It was important to maintain commit history since we will eventually be retiring Bazaar and we don't want to reference Bazaar's revision history.

In comes Windows Subsystem for Linux (WSL):

With Windows Subsystem for Linux, I was able to get Ubuntu 18.04 LTS installed and setup Bazaar and Git. When running Bazaar from Ubuntu 18.04 LTS in the WSL, the migration completed successfully. The Bazaar process had 64bit process memory available and didn't fail from being out of memory. Setting up the WSL took about 10 minutes and the full migration process took about 10 minutes. Within 20 minutes I was ready to push to our Git server after the migration.

Retro Video Game Homebrew

3. January 2019 07:24 by Cameron in   //  Tags: , , , , ,   //   Comments

Over the past few months, I've been watching various videos about the cool potential of older gen six/seven consoles for homebrew and modding. Most of my inspiration comes from ModernVintageGamer, but I have been following a few others as well (Adam Koralik and Dreamcastic for Sega Dreamcast). Since my last post, I've been collecting older gen six/seven consoles including the original Xbox, Xbox 360, PS2, PS3, and Sega Dreamcast. My end goal is to have a working development environment for each system and try to port some homebrew to each platform. I would like to make a simple 2D game engine that can run on all of these systems. I have a lot of work ahead of me, but it will be a good learning experience to get familiar with each system's architecture. From what I've read, the hardest console of these to develop for will probably be the PS2, but I will have to wait and see. Fortunately, all of these systems are old enough that there are mature homebrew communities and SDKs available.

Getting HD-DVD movies into Plex

28. September 2018 09:31 by Cameron in Home Theater  //  Tags: , , ,   //   Comments

In order to get HD-DVD movies into Plex you need the following:
1. External/Internal HD-DVD drive (an Xbox 360 HD-DVD is cheap on eBay ~$10-15)

2. MakeMKV 

3. AnyDVD HD/DVDFab Passkey (used for decrypting HD-DVD discs)

4. Clown BD (used for converting decrypted HD-DVD movies to M2TS)

5. Plenty of hard drive space as movies can be on average 10-20GB a piece. I use a Synology DS918+ NAS with 12TB of space.

Method 1:

The easiest approach to getting HD-DVD rips is using MakeMKV and selecting the main feature to rip. The process is similar to ripping a standard DVD or Blu-Ray/UHD disc. Just insert the disc in the drive and wait for MakeMKV to recognize the disc. Then load up the title list and select the main feature (the largest title in the list). In terms of audio, most discs have Dolby Digital Plus. Some have TrueHD which is lossless and preferred if available.

Note 1: Some people have mentioned that using an older version of MakeMKV (1.9.9) has had better success than more recent versions. You can try this if you have read errors on some of your discs.

Note 2: The mkvs produced by MakeMKV do not remove the telecine pull-down flag so you are left with a 29.97 fps video. This may not be an issue, but if you want to remove this flag, you will need to demux the mkv using eac3to and then remux (replacing the original video) using something like MKVToolnix.

Method 2:

If MakeMKV can't read the disc, then you may need to try with AnyDVD HD/DVDFab Passkey and rip the entire video disc to a temporary location on your hard drive. Then you can use Clown BD to demux/remux into an M2TS.

Method 3:

Use AnyDVD HD/DVDFab Passkey to take a full ISO backup of your disc. Then mount the disc and attempt Method 1 or Method 2 (point Clown BD to your virtual drive instead).

If these methods are unsuccessful, you might have disc rot and be unable to fully read the disc. In this case, you can evaluate if you want to replace the damaged discs with their Blu-Ray release.

General Notes:

Audio in Plex is a bit finicky depending on your client. If you use the PC client, then nearly any format is supported via direct play. However, if you use a client such as Xbox One or PS4, these might need to transcode Dolby Digital Plus to AAC before being compatible for playback. I found that on my Synology DS918+, this can be troublesome because transcoding DD+ to AAC requires a lot of processing power and sometimes the movie buffers while the audio transcodes.

The bottom line for movie enthusiasts:

Be smart about adding HD-DVDs to your library. Warner Brothers titles are more risky and it's a bit of a mixed bag with them due to disc rot. I have had success with a few titles so far, but I'm not holding my breath on all of my Warner Brothers titles. Also consider that since HD-DVD has been out of commission for the last 10 years, movies will only be as new as 2008. However, there are dozens of titles from that era that are worth collecting.

Why I bought an HD-DVD player in 2018

28. September 2018 09:15 by Cameron in Home Theater, server  //  Tags: , , , , ,   //   Comments

I recently stumbled upon a video on YouTube about adding various upgrades/addons to an Xbox 360. Among these upgrades included the Xbox 360 HD-DVD drive. Since HD-DVD lost the format war against  Blu-Ray back in 2008, it is considered a dead format and likely any movies on the platform are cheaper than their Blu-Ray counterpart. That's when I had the thought to purchase this drive and some HD-DVDs to expand my HD movie collection. Because the drive communicates over USB, it can be used with a PC and you can either play the discs directly or rip them to play them on your media server. I went to eBay and found a 46 movie lot with two HD-DVD drives for $60 which is a huge discount on movies.

I am now in the process of converting these movies with MakeMKV and AnyDVD HD/Clown BD. Most of the discs I've tried have worked with MakeMKV, but a few might need a different approach. I've been able to rip most of my library so far without issue, but some of the Warner Brothers titles are more difficult to rip due to their impending disc rot. Luckily, I was able to rip "A Clockwork Orange", "I am Legend", and "Constantine" so far and there doesn't appear to be any disc rot on these. I have about 10-15 more Warner Brothers titles to assess, but hopefully I can get through most of them. Disc rot is inevitable on Warner Brothers HD-DVDs so if you have any of these still, get them backed up ASAP!

I will be looking for single movies or smaller lots that don't contain Warner Brothers movies in the future. However, even if only 20-30 movies out of the 46 are good in this lot, it still is a heck of a deal on all of the movies.

HTPC Upgrade/Rebuild!

28. July 2018 22:55 by Cameron in Home Theater  //  Tags: , , , , , ,   //   Comments

I've upgraded and nearly completely rebuilt my HTPC for better long term usability and more moderate gaming. I swapped out the motherboard with an HP Elite 8300 motherboard, upgraded the RAM to 16GB, upgraded the CPU to an i7 3770, upgraded the SSD to 240GB, upgraded the hard drive to 4TB, and upgraded the GPU to a GTX 1050 Ti.

Originally, the upgrades were inspired by high CPU usage when playing H.265 content from Plex. I noticed this when playing back 4K H.265 content from Plex on my i5 2500. With the i7 3770, I am able to play back 4K H.265 movies with roughly 30-40% CPU usage. Adding a GTX 1050 Ti also helped with the decoding via its CUDA cores. The GT 1030 simply didn't have the power to help with H.265 so everything was CPU bound. I needed to swap motherboards because the HP 6200 Pro doesn't supply the needed 75W for the GTX 1050 Ti. Since I was swapping motherboards, I went for the HP Elite 8300 so I could get Ivy Bridge CPU support. Naturally, I needed to get the best i7 available. The larger SSD was just so I could install a couple of higher demanding games such as Rise of the Tomb Raider and Just Cause 3. I outfitted the rig with a 4TB drive to accommodate installing more Steam games and support recording TV shows/movies from Plex DVR.

As a result of these upgrades, the only thing original to the PC I first bought is the chassis. Now, I am happily using this machine as a more moderate gaming machine and my home theater workhorse. I will likely upgrade the low profile GPU as needed in the future, but the GTX 1050 Ti should suit me for a while.

Plex with Synology DS918+ NAS for home media

28. July 2018 22:19 by Cameron in NAS  //  Tags: , , ,   //   Comments

I realized that I was going to quickly outgrow my desktop PC's storage for use with Plex once I started ripping my 4K UHD Blu-ray disks. To remedy this, I recently purchased a Synology DS918+ NAS for handling storage of my files and TV/movie collection. I purchased the DS918+ because of it supporting four drives, expandability options (expand up to 5 more drives), and its processor for Plex transcoding.

To start, I installed three 4TB Seagate IronWolf NAS hard drives and configured to use Synology's hybrid RAID. I will be adding a fourth 4TB hard drive later this summer. With the hybrid RAID option, I can have mixed hard drive sizes in my array and upgrade the disks gradually. The main requirement is that I have two of the largest disks in the array. I will probably upgrade all of these drives to 8TB at some point, but it shouldn't be right away.

I have been thoroughly enjoying Plex and having my TV shows, movies, and music available anywhere in the world. I ripped all of my audio CDs with iTunes and copied over all of the albums I bought on iTunes too. I also bought an HDHomerun Prime to record cable TV and save to the NAS. I record with my home theater PC and strip out commercials with MCEBuddy and Comskip. After recording, MCEBuddy copies the files to my NAS for archiving.

I am strongly considering cancelling Hulu and Spotify in favor of my own libraries. I think Hulu is less necessary when you can record your own TV shows and watch them at your own leisure. In regards to Spotify, I hardly listen to much of the new stuff out there and it would be cheaper to buy an album on occasion to support individual artists. Spotify doesn't pay artists very much for music played.

It takes a bit of patience to rip a large movie collection. However, all you need is a PC with a Blu-ray drive and MakeMKV to save the rips directly to your NAS. You can rip everything from DVD to UHD Blu-ray with MakeMKV. Ripping UHD Blu-ray is more challenging as it requires specific drives with specific firmware. However, the process isn't too bad after you've patched your drive's firmware (assuming you have a newer drive). Ripping audio CDs is very quick since they contain at most 700MB of data.

With Plex for Windows, I am able to play UHD mkvs on my 4K SDR TV with HDR to SDR tone mapping. As far as I know, the Windows application is the only client capable of tone mapping. Most UHD releases also come with 1080p releases, but I must say that the tone mapped colors are much more vivid than their 1080p SDR equivalents. I jumped on the 4K TV bandwagon before HDR was readily available, but I will be getting an HDR10/Dolby Vision TV at some point in the near future. At that point,tone mapping will not be needed.

Since getting all of this setup, I have been extremely happy with my purchases. There is a bit of an upfront cost with all of the equipment, but the features in Plex make this worth every penny. Whether you're just beginning to build a movie library or you've been collecting a while, Plex is a good way to get your collection preserved and make it available anywhere in the world.

Tevo Tarantula Update!

24. May 2018 10:42 by Cameron in 3D Printing  //  Tags: , , ,   //   Comments

It's been nearly 2 months since my last update, but I have been busy printing and tweaking! I have printed a few gifts for my family (small jewelry boxes) and I've printed some upgrades to the printer. Since my last post, I installed the dual-y rail upgrade and began using PEI as my build surface. Since the dual-y rail upgrade, my bed has been much more stable. I am also running my printer from an MKS Gen L board with TMC2208 drivers. The TMC2208 drivers are drop in replacements for the stock A4988 drivers and much more efficiently than my previous DRV8825 drivers.

I think there is something strange with my frame though as my prints are just a slight bit slanted. I will likely need to do some frame stabilization upgrades in the near future to resolve this issue. I will likely go with more aluminum extrusions in order to ensure that my frame is rigid and straight up and down. I have 3d printed spacers for the dual-y rails, but I think these aren't perfect so I would like to fix this with a frame upgrade.

Printing on PEI has been a dream come true. I no longer have bed adhesion issues. I have only been printing PLA on PEI so far, but I will be getting into trying PETG soon. I had printed PETG directly on a sheet of borosilicate glass previously, however, the PETG bonded too strongly with the glass and when I removed the part, it took small chunk of glass. This chunk of glass was very small, but if I continued printing PETG that way, I could have ended up with a very uneven build surface. I have heard that PETG sticks very hard to PEI, but I will need to do some experimentation to see!

My next upgrades include a square frame (full base) with right angle mounts for my dual y-axis rails, stronger center brackets (replace the flimsy tiny metal brackets), z-axis supports, and potentially some feet out of TPU to absorb vibrations from the printer. My prints are currently at an acceptable quality level, but I do know, with the improvements I listed, my print quality can improve more.

At this point, I've spent enough money on the upgrades for this printer to where I could have just bought a Tevo Tornado or equivalent. However, these printers are more or less put together when you unpack them from the box. I have learned far more from putting together my Tevo Tarantula than by simply building a 90% complete printer.

My previous printer, the Monoprice Select Mini v2, although a good printer for most, is difficult to service (for beginners). I was, at the time, very much a beginner at 3d printing. It was a bit intimidating to take apart the Monoprice Select Mini v2 since I knew nothing about the internals of 3d printers. With my Tevo Tarantula, the entire build process with upgrades has been monumental in my understanding of 3d printing.

At some point, I would like to get a second 3d printer, but I will likely hold out for a full color 3d printer which mixes colors in realtime. At this time, full color printing is more of a luxury with a printer or extruder upgrade + printer costing upwards of $1000. We are on our way of this becoming more mainstream in the next year or so. For now, I am happy with my dual color/dual material Tevo Tarantula. In the meantime, I may explore painting my prints in post processing.

Update on Tevo Tarantula!

7. April 2018 16:27 by Cameron in 3D Printing  //  Tags: , , , , , , , , , , , ,   //   Comments

It's been a little over a month since my last update on my Tevo Tarantula. All of the previous planned upgrades are now installed and I am fine tuning to get high quality prints.

My dual extruder and large bed upgrade kits arrived the last week of March and I got both configured in Marlin and reflashed the firmware. The large bed upgrade was easy enough once I moved the Y-endstop to the back of the aluminum extrusion and adjusted the X-gantry position. Once I had everything heated up to test my second extruder, I noticed that my second extruder driver wasn't tuned correctly and the motor was clicking and didn't extrude. I adjusted the voltage on the E1 driver (stupidly without a multimeter) and eventually I turned the potentiometer too much and it ultimately broke. Since I had the stock MKS 1.4 base board, the stepper drivers were integrated, and I couldn't replace them. Because of this, I had to order a replacement board if I wanted to use the second extruder.

I wanted to avoid issues with stepper drivers in the future and I ordered an MKS Gen L board which has replaceable drivers. I also ordered two sets of DRV8825 drivers (5 pack) so I could be sure to have backup drivers in case I needed them. As it would turn out, I did need them! I didn't realize that the drivers were cranked up to 1.6v when I powered on the printer. I burned through 3 of the extra 5 drivers before realizing I needed to configure the jumpers for each driver and adjust the voltage down to 0.75v (this time with a multimeter).

These DRV8825 drivers run much quieter than the stock A4988 drivers and before I replaced the drivers, I had installed stepper motor dampers on the X and Y axis. Now printing is extremely quiet! As a future upgrade, I am going to replace all of the stepper drivers with TMC2208s which should make the printer virtually silent aside from the fans. Maybe I can replace the fans at a later point to reduce noise. The great thing is that I can now run prints at all hours of the night and day with minimal noise.

I decided I would go for the dual-Y rail upgrade to provide more stability for the build plate and reduce ghosting in my prints. I still need to print all the necessary components for this upgrade, but it will be a huge improvement once implemented. After I install the dual-Y rail upgrade, I'm hoping this will be the last upgrade I need to make on this printer for a while. Of course, I will always be tuning little bits along the way to improve quality. Originally, I got into 3d printing to enable me to print enclosures for electronics projects for Raspberry Pi and Arduino. However, this has evolved way past that and I'm thoroughly enjoying the process of building/tuning the printer and creating neat prints. Many friends and family members have asked me to 3d print various objects for them as gifts or toys. I think that is really cool and I'm happy to do that for them. I've also helped troubleshoot issues on friends' DIY 3d printers with the knowledge that I've gained from working on my printer.

One of my latest prints before my last rebuild was of the Millennium Falcon (named Fillennium Malcon on Thingiverse). I am pretty happy with how it turned out and it now sits on my desk at work. I printed the model at 60% scale with Cura. It took roughly 4.5 hours to finish printing. I had one failed attempt trying to print at 40% scale and about 70% into the print, the part fell over onto the build plate. This was due to my not printing a proper raft. On my second run, I made sure to print a large raft to avoid the same thing from happening.

I will soon be looking into PVA as support material for more complex models with dissolvable supports. I just need to see about how to properly store PVA since it is very sensitive to humidity. Perhaps I can invest in a single spool enclosure for the PVA. Cura's support options for PVA are somewhat limited meaning that it can only designate supports to one extruder. This will result in using a lot of PVA to support larger objects. Simplify3D and Slic3r Prusa Edition have an option to print PVA on only the areas that touch the model and your primary material for the rest of the supports. I am still deciding whether I want to buy Simplify3D when Cura suits 95% of my needs for everyday printing. At the price tag of $150, it better have pretty compelling features for me to make the switch from Cura. If I see myself burning through my supply of PVA quickly, Simplify3D would pay for itself in the long run. Rolls of PVA run for $35-50 which would be less expensive over a long period of time with Simplify3D.

Update on Tevo Tarantula

3. March 2018 17:16 by Cameron in 3D Printing  //  Tags: , , ,   //   Comments

The dual-z axis upgrade kit came in this past week and I've been working on tuning the two lead screws to move together. This has been slightly challenging since I have two z-axis motors which can move out of synchronization. This results in having to adjust the Z-offset on my hotend occasionally and re-calibrate the bed for better first layer adhesion. Since the lead screws are not perfectly aligned up and down, there is quite a lot of vibration which causes a loud screeching sound during bed leveling and homing. I have tried using a hand level to get the lead screws aligned properly, but this has been difficult.

Instead, I am going to try using a single motor pulley system to drive both lead screws. I will need to print new brackets for this setup. The motor is mounted inverted which will hopefully decrease the wobbling. I was able to source most of the parts needed for this upgrade from the United States. The only piece I couldn't find was the closed loop GT2 timing belt. I ordered two new lead screws in case either of the ones I had from Tevo were accidentally bent slightly. I am hopeful that this will make my z-axis virtually quiet and will also improve my print quality immensely.

I am using supasorn's belt tensioner and dual-z axis design from Thingiverse. You can find his profile here: https://www.thingiverse.com/supasorn/designs

On both of his belt tensioner and dual-z axis pages, he has pictures of flawless 20mm calibration cubes. This is my ultimate goal after making these upgrades.

I ordered a dual-extruder kit and large bed kit from Tevo and hope to see those by late March/early April. In the meantime, I will be printing/tuning around the clock to get my workspace more organized. My desk is just covered in electronic boards and wires at the moment. I did finally manage to print the TUSH filament holder so I no longer have a ghetto filament holder setup.

Since I have my printer working well enough now, I thought I would try and print Maker's Muse Lattice Cube design. I made two separate scaled prints of the cube. The first attempt at printing the lattice cube did not work too well. I had scaled it to 25% in Cura, but the print came out a bit rough. There wasn't much room for the nozzle to move around freely at that scale. The cube nearly completed, but the nozzle got stuck on the top of the cube, near the end of the print, and I had to abort the print. In my second attempt, I scaled to 40% and printed with my silver PLA. The print took roughly an hour and the result is quite nice! After I tune my printer a bit more, I may try some of the more advanced designs.

I ordered some more colors of PLA filament (yellow, red, green). I would like to print all of the low poly designs (namely Pokemon to start) and I will be collecting different colors of PLA over the next few months. I would like to try some other materials soon, but I am not sure whether I want to try PETG or ABS. I am interested in flexible filament, but I'll need to modify my extruder to support those. I will be investing in PVA at some point in the near future too.

That's it for now! I will continue to post updates as I make more progress. I am going to work on a beginner's guide to building/configuring/upgrading their Tevo Tarantula soon.

Progress with my Tevo Tarantula

11. February 2018 15:30 by Cameron in 3D Printing  //  Tags: , , , , ,   //   Comments

It's been a couple of weeks since my last update on the printer. Since my last update, I have received my aluminum brackets and replaced all of the original acrylic parts with their aluminum counter parts. The build is much sturdier than before. However, I have found that it is harder to get the bed manually leveled due to the rigidness of the frame. I'm in the process of getting a BL Touch sensor installed to allow for a less strict leveled bed. This should improve print quality and not require me to level the bed anymore. Previously with the acrylic parts, I could get away with less of a leveled bed and get high quality prints.

I've been trying to print pieces for my electronics enclosure, but the prints keep failing due to the large surface area and the bed not being perfectly level. Most of the time, the print would be fine until close to the end and then there would be some layer shifting that ruins the print. This is unfortunate because the part I'm trying to print takes 6-7 hours to print and most of the time I've set it to print over night. When I come back to the print in the morning, I find that the print has failed. I'm fairly confident that installing the BL Touch sensor should fix most of these issues.

I'm currently working through which version of the community firmware to install on my printer. I have Jim Brown's Easy Config based on Marlin 1.1, but for some reason, the BL Touch doesn't appear to work correctly for the Z-endstop and when I do the auto home mechanism, the print nozzle crashes into the bed. I'm going to try a few other firmware configurations available from the Facebook group. There are a few guides referring to the BL Touch and specific firmware releases. I'm fairly certain the BL Touch is not at fault because when I power on the printer, it deploys/retracts the sensor a few times. I expect that it's a firmware configuration issue and I should be able to get it working in the next few days.

I have a seconds Z-axis rod/motor kit coming from Tevo in the next couple of weeks. That should help with Z-axis stability and print quality. Most people say this is not necessary, but I don't think it will hurt. Once I have the BL Touch installed, I will do some before/after comparison prints with/without the dual Z-axis kit. I've also ordered a large bed sticker from Tevo to apply to a large bed that I will purchase from Amazon. I'm attempting to stay as stock as possible, but the kits from Tevo tend to be a bit more expensive and they take longer to ship from overseas than from a seller in the US. Eventually, I want to get a dual extruder upgrade, but I'm not sure if I will go with Tevo or source the parts separately from Amazon. I found a dual extruder kit on Amazon that is roughly half the price of Tevo's, but I would need to get a second extruder motor and part assembly. Because of the price of the individual components, it might just make more sense to get Tevo's kit. I'm mainly interested in printing complex models using PVA (water soluble) for supports and PLA/ABS/PETG for the model.

I will continue to post updates on the print quality and upgrades soon!

Month List

Tag cloud