Like I promised, here goes a blog post to show off the finished product.
Now I am going to print a box for my temperature reading raspberry pi board, that fell apart yesterday when I was connecting/disconnecting cables from it. But first got to find a model of one.
Hello again fans,
Since I took today and tomorrow off work (small vacation) and given the fact that I am forced to stay at home because of COVID-19, here goes another blog post.
Tonight I am 3D printing a coaster for holding my beer when I am working remotely from home. Pic below (still couldn’t see anything, took the photo on the beginning of the process).
Also, it has been a while since I came down to my garage to continue the several projects going on. The featured image on the top of this post is of my to be plotting machine. I am currently testing the motors, that are controlled by an Arduino with a shield on it.
And not forgetting about the temperature sensing device, that is almost done, I just have to make a permanent circuit for it. Picture below :
Well, that’s it for now. When the coaster is done printing I will make another post with a picture of it.
I get a lot of junk comments on this site, posted by bots on the net that try to do phishing attacks. They don’t get displayed on the production blog, because I never authorise them. Right now I have over 35000 comments in the database, that I need to weed out as being spam or not.
I recently was reading an introductory book on data science, and that book has a chapter where a method for spam filtering is illustrated. So I set about to implement it and see if it works on my data.
I first had to copy all the messages on the main comments table in the database to another table that serves as the training set. Then Over each word on each message of that set I calculated the probability of it being spam or not spam, when seen in a message. Note that I considered all words to be spam in the training set, since I don’t get any legitimate comments, because basically no-one reads this site.
Last step, when receiving a new message I calculate the probability of it being spam given the word probabilities calculated before. Right now I have a job running periodically processing the backlog of messages I already have in the database. I am curious of seeing what will be left of them after I process them all.
My retro collection is growing. This amiga was bought from a guy in the UK. I have opened it and saw that the RF metal shield is a bit rusty. Other than that it works mostly well. I still don’t know if the PCMCIA slot works, because there are compatibility issues with PCMCIA to compact flash cards.
I installed a 8 MB RAM expansion on it. You read it well, 8 MB ! Removed the mechanical hard disk that was installed, and put a IDE to compact flash adapter and a 32 GB card on it. Still can’t use all of the capacity on it, but I am still learning how to. Below is another shot of the machine.
I got a SCART to HDMI adapter (the little black box show in the pic above) so I could connect it to my LCD TV. The picture is not the sharpest but its ok. These machines were mostly used for games back in the day, but I am interested in them as development platforms. From what I have learned, most software made for them was developed on other systems and then cross compiled for it. I got a C-Compiler for it, and a book on basic OS programming, so I have enough to get started. I really want to code in the actual machine, or else I could do it on a PC and run the programs on a emulator.
My recent acquisition of a zx spectrum+ from an italian seller didn’t go very well. He said the machine worked and it was not true. So I decided to lash out some more money and bought a refurbished zx spectrum main board, to fit inside the case of the faulty one.
I received the board in the email plus a power supply, and assembled it. When I went to test it, the keys were not working. So I had to buy a new keyboard membrane online. More euros down the drain.
Finally the machine seems to be fully working, only to find out that one of its rubber feet is damaged, so if the computer wobbles a bit it restarts :). Solution was to put some papers under it, and it wobbles no more.
Today I had a shot at the original 3d isometric batman, the one where you search for the pieces of the bat mobile. I didn’t get very far. Need to practice more. It brought back the memories of me, 12 years old playing on one of these machines (a timex 2048 made in Costa da Caparica, Portugal). I literally spent days playing on it, until I became a juvenile delinquent at the age of 15.
It was quite some time since my last post. Since I printed my first 3D object, a lot has been done. Xekmypic, my picture private cloud is generating data to be analysed by a set of small hadoop programs. It has been fun doing it. Still some more to do before I move on to other more sophisticated ways of doing things.
Other than that, I have been gearing up my garage for electronics stuff. An oscilloscope, a function generator, small parts (like resistors, capacitors, etc).
I then got the urge to get into the retro scene, and therefore i bought a zx spectrum + off eBay from an italian seller. He said the machine worked, only for me to find that that was a lie. I could ask for a refund, but I prefer trying to repair it myself. There are some resources on the net related to the problem that the machine has.
I am also doing some stuff with an arduino and trying to read a temperature from a sensor. The deal is this : an arduino has a shield that has a wifi chip (the ESP8266) that can make a hotspot and serve data over TCP connections. So I have a RTD temperature sensor, connected through a circuit to the arduino with the wifi shield. The arduino periodically reads the value of the temperature, and publishes it on the TCP service.
I then have a raspberry pi 3B that has a TCP client implemented in python, that constantly gets the temperature from the arduino and stores it in my database that is in my apartment.
Next, I will try to make a web page that displays a graph of the values collected along with a mean value, because the temperature readings oscillate a bit.
I have some other things in stock. Stay tuned.
So I am now at a point in the development of Xekmypic, where I must start sending data to hadoop. I already have a bot that simulates most end-user actions on the client api. This api will be modified to send data to hadoop when certain events occur.
But the sending process is not straightforward. The deal is to publish a message with the event data on to a queue in a rabbit MQ server. Then a custom application (Java) will pickup messages from the queues and call the HDFS apis to append to the current file being written.
There are a lot of implications on this consumer application. My idea currently is to have a pool of consumers, that block on the receipt of messages. When a message arrives, a thread is spawned with the code that will write the event data to HDFS. Some questions arise here. Is the appending process in HDFS thread safe ? Can I open a connection to hdfs to do filesystem operations and leave it open for long periods of time ? I need to have some hierarchy of folders in HDFS, so that I don’t get too many files in a single directory. Should I start appending right from the beginning of a file, or should I write to the normal file system up to a specific size and then append it to the HDFS file ?
As promised Here is a post with a photo of my first 3d-printed object. It’s a useless object, but I needed to test the printer for any defects since I bought it recently.
I was up all night because the print lasted about 10 hours and I started printing yesterday at 11PM. Why didn’t I let the printer do its job and go to bed ? Because from what I’ve read on the net there is a fire hazards on these printers (they have heated elements) and they should not be left printing unattended.
So the first print went ok. I like the quality of the texture of the plastic surface, and next will be printing a part of my house’s heated elements. Its a plastic piece that holds the heater to the wall. It broke. I will try to create a 3d model from photographs, and go from there.
I am now the proud owner of a 3d printer. Haven’t set it up yet, it needs calibration of several of its moving parts. In the pic you can see it along side of the PC in my garage.
When I print something I will create a post here to let you know.
I am trying to automate the generation of user traffic on my big data test application (xekmypic). I followed the approach of automating the mac osx client through applescript, but it’s not cutting it. Besides taking over the whole operating system focused on the open and focused application, random errors occur related to apple events.
So I am in the process of writing a client bot in python. I will then be able to spawn several processes of this script and get better throughput in usage generation.
One issue that arose is the generation and exporting of usage events to the server application. This is the most critical part of the simulation, because event data will be the source data to analyse with big data tools. I cannot generate the events and export them immediately, I have to mimic the way the events are exported by the desktop client. Which is storing the events in the database as they are generated, and periodically have a task that exports them.