Browsing all posts tagged web

Now that I am fully managing this website's back-end, I've had to educate myself on configuring nginx. Here are a few of the things I've learned since starting down this path:

Using try_files

At first, I couldn't figure out the magic for having nginx serve a static file if it was actually present on the server, but fall back to my Django application if it wasn't. The answer is to use try_files, a handy built-in resource for this exact use case. I found this via this helpful entry in the Pitfalls and Common Mistakes article (which itself is a treasure trove of information). The specific entry ended up looking like this:

server {
    location / {
        try_files $uri @proxy_to_app;
    }

    location @proxy_to_app {
        # Typical proxy-pass stuff here
    }
}

Shortcutting Known Bad Paths

Don't let your Django app handle paths you know are bad; let nginx offload that processing work for you! I was seeing plenty of accesses against .php files in my server access logs, so I wrote a quick block to reject them:

server {
    location ~* \.php$ {
        return 404;
    }
}

Enabling Strict-Transport Security

Until recently, I didn't even know that HTTP Strict Transport Security was a thing, but it turns out to be pretty easy to implement. Once you have your SSL stuff ironed out and working, you simply need to enable the appropriate header:

add_header Strict-Transport-Security "max-age=5184000; includeSubDomains";

Online Radio Scanners

Mar 15, 2024

What a world we live in.

A few moments prior to my writing this, I heard fire engine sirens outside my house. Looking out my office window and through our neighbor's trees, I could see multiple units responding to a house across the nearest main street. I wondered to myself: "Could I determine what problem they were responding to?" A quick google for "Raleigh fire scanner" returned several hits, one of which was OpenMHz. I pulled up the site, saw that I could filter on "EMS Fire Alerts Only," and selected that option.

Clicking backwards through the playlist, I found the corresponding entry: a gas leak! Cross referencing the reported address, I verified that the house was indeed near our neighborhood. Success!

It's amazing what is possible through the internet. As an aside, this website appears to be a really neat way to listen to what's going on in the area. I can listen to the airport (there was a reported ash-can fire this morning), Raleigh police, or a general overview of all channels (a multiple-vehicle crash occurred a few minutes ago down near Fuquay Varina). Other filters are also available; pretty neat!

Django Web Hosting?

Feb 2, 2024

The web host I've used since the inception of this site will be dropping support for Python (and by extension Django) based applications at the end of March. As such, I'm wondering what alternatives exist. Is anyone familiar with a host that supports Python / Django applications? I'd appreciate any suggestions.

Radio Garden

Oct 20, 2022

Radio Garden is a particularly interesting website that allows you to browse live radio around the globe. Functioning a little like Google Earth, you're given a satellite view of the planet. Littered across the globe are little green dots, each of which represents one (or more) radio stations based in that location. Move the crosshair over a specific dot, and you'll hear the live stream of that radio station.

This is a really neat way to "travel" around the world. I've heard local news reports from Alaska, ethnic music in various African countries, and content in all imaginable languages. I've found it humorous how many stations outside of the United States play American or British bands (Queen, Pink Floyd, The Beatles, The Beach Boys, and similar acts can be heard all over). This is a neat way to spend some time, and it makes for a great way to listen to music while working.

Particularly interesting are all of the stations in really out of the way places. Hear what folks are listening to in the middle of the Pacific Ocean!

On Human Curation

Jul 27, 2022

A recent article at The Verge entitled Adam Mosseri confirms it: Instagram is over got me thinking about content curation. One of the article's arguments revolves around how "the algorithm" is partially to blame for Instagram's slow demise. I'm not an Instagram user, but I do use YouTube, which has similar problems. The home page of YouTube is skewed by what "the algorithm" thinks I want to see. Most of the time, it's surprisingly bad at predicting what I might be interested in. One of my major gripes is that it often suggests things I've already watched.

What I'd love is for more platforms to offer human curation. Something along the lines of kottke.org (which I happened to be a late-comer to; kottke.org is currently on hiatus). I claim that human curated content, done correctly, would outperform any algorithmic means currently employed.

Live Flight Radar

Jun 26, 2022

Recently, while sitting out on my back deck with the kids, I wondered if it was possible to identify the planes flying over my house. (We live near the Raleigh-Durham international airport, which means there are always planes visible). I asked Brave if this was possible, and found out that it was!

The Flightradar24 website allows you to view flight paths of planes in real time, which is so neat. They have an associated app, which I downloaded to my phone. I can now see a plane, pull up the app, and identify the flight (where it's coming from, where it's going to, etc.). The app shows big commercial flights, as well as smaller private flights. Helicopters are also displayed. What a neat world!

Brave Search

May 18, 2022

For the past few weeks, I've been giving Brave Search a good college try. I've been doing this in an effort to reduce my dependency on Google, as well as to reduce my exposure to their advertising and profiling mechanisms. So far, I've been pleasantly surprised. The majority of my searches are of a technical nature (usually in regards to my work), and the search results generally have exactly what I'm looking for. Their recently added Discussions feature has been particularly helpful to find results from sites like Stack Overflow and Reddit.

It isn't perfect, however; a few recent searches on some esoteric technical topics (7-zip performance in Ubuntu, for example) left me mildly disappointed. However, I've been pleased enough that I think I'll start using this as my go-to search engine. It works great in the Brave browser, which I am also now using as my primary driver.

Is it just me, or has the pin density on Google Maps been upped considerably? Take a look at the following image, centered on the I-540 / I-40 interchange here in Raleigh:

Google Maps pin density example

There must be close to 50 or more pins in this (fairly zoomed out) view. The vast majority of these pins are places I've either never visited or don't care about. Maybe this is a way that places can "advertise" with Google? I would argue that this density of pins makes the map incredibly hard to read. I feel as though Google Maps used to be way less busy than this.

During the deploy process of my new web platform, I ran into a really puzzling situation. I had my application deployed per DreamHost's documentation, but I kept getting the standard domain landing page ("borngeek.com is almost here!"). This deploy was set up exactly like several other sites I had working successfully, but this one just wasn't working.

I ended up having to reach out to DreamHost support (which is fantastic, by the way), and one of their technicians helped me figure out the issue. Prior to deploying this new Django-powered platform, I was running with WordPress. While running in that environment, I had enabled HTTPS support through the Let's Encrypt service, something DreamHost offers for free. The HTTPS service was configured with a path that no longer applied, seeing as my new folder structure is different from the old WordPress setup.

There are two ways the pathing issue can be fixed at DreamHost:

  1. You can reach out to the support team to have them ensure that the HTTPS service is pointing at the right location.
  2. You can remove the HTTPS certificate from the domain, and request a new one. The path will be updated as a byproduct of this change.

Hopefully this tip will help someone else out there, as I spent several frustrating hours debugging this issue.

Spam Mitigation

Dec 18, 2020

One of my primary concerns in writing my own web publishing platform was how to manage spam. In the WordPress world I used both Akismet, as well as another third-party plugin, to keep things under control. In looking around at various options, I stumbled upon a terrific article at Ned Batchelder's blog on how he manages spam.

His technique can prevent both playback bots, as well as form-filling bots, from submitting garbage data. The process is fairly basic:

  1. A timestamp field is inserted as a part of the commenting form.
  2. A spinner field is included, its value being a hash of four key data elements:
    • The timestamp
    • The client's IP address
    • The entry ID of the post being commented on
    • A secret
  3. Field names on the form are all randomized, with the exception of the spinner. The randomization process uses the spinner value, the real field name, and a secret.
  4. Honeypots are scattered throughout the form, and made invisible to humans through CSS.

Once the form data is submitted, valdation occurs to detect whether a bot was present:

  1. The spinner gets read to figure out which form fields match which data.
  2. The timestamp is checked and rejected if it's in too far into the past, in the future, or not present.
  3. The spinner value is checked to ensure it hasn't been tampered with.
  4. Honeypots are checked to see if data is provided in any of them.
  5. The rest of the data is validated as usual.

Ned's article goes further into the details than I have above, so I highly recommend reading it if you're interested in this kind of thing. Time will tell as to whether this technique will be successful at keeping bad comments out, but I'm optimistic that it will.

One of our local news stations recently deployed an anti-adblock package (Admiral) on their website. It detects the presence of adblock software on the client, and prevents access until you white-list the website. The ads shown by this particular website have previously included malicious ones that attempt to deploy malware. As such, I refuse to white-list their site, having had bad experiences in the past.

While searching for tactics to sidestep this, I stumbled upon two Reddit threads (thread 1 and thread 2), both of which gave me enough information to figure out what was going on. The second thread above points to a repo of domain names used by this third-party solution for serving their adblock detection software. There are enough similarities in the domain names they use that make it pretty easy to pick out patterns. A typical pattern they employ is:

  • <adjective><noun>.com

This is similar to how default Docker containers are named. An example is unequalbrake.com, which also happens to be the domain serving the aforementioned news website instance.

Adding this domain (or list of domains) to your adblock filter list should block the adblock blocker.

NewsBlur

Jan 9, 2019

I consume nearly all of my news and web reading through RSS feeds, and have done so for many years. Back before July 2013, I used Google Reader, before Google shut it down for good (the bums). Shortly after Google Reader was closed, I switched to NewsBlur, which closely resembles the Google Reader of yore. I cannot imagine surfing the web without it.

The service can be used for free (up to a maximum of 64 feeds, and only 5 stories at a time in the "river of news" mode), but I pay a yearly fee ($36) to have an unlimited number of feeds and stories.

One of the best features of NewsBlur is being able to "train" the reader to know what you like and don't like from various sites. I've trained several of my gaming feeds to exclude anything about Fortnite, because that's a game I have no interest in. As a result, stories on that topic are never shown to me; I'm in control of what I read! Similarly, I've also trained a few feeds to flag articles from authors I like. Those articles are highlighted, and I can view just those highlighted ones if I so choose with the "Focus" mode.

There are lots of other features to recommend about NewsBlur: searching for stories across all of my feeds, sharing stories with friends on the service, a pretty nice Android app, and lots more. If you haven't given NewsBlur a try, be sure to.

Remember the days of Microsoft FrontPage? I first cut my web development teeth using that tool, and at the time I thought it was amazing. Designing a web site was made easy and I really liked the WYSIWYG editing style. I eventually migrated to Adobe Dreamweaver which seemed (and was) even more powerful.

Nearly 20 years have passed since my start in web development. With a computer science degree under my belt, along with 13+ years of professional experience, I can only look back on those days and laugh at my naivety. Those tools seemed slick at the time, but they were pretty clunky in actuality. The HTML and rudimentary CSS that each generated was ugly and bloated. That said, the WYSIWYG movement never really died. With modern companies like Squarespace and Wix.com, the "build it as you go" web model is still alive and kicking.

WordPress now also seems to be headed that way. I use WordPress here at Born Geek, and I just recently updated to version 5.0. The giant new feature in this release is the new Gutenberg editor, which offers a visual means of laying out your content. To a technical minded person like myself, who typically writes posts in Markdown, the editor is incredibly confusing. I don't want to have to insert "blocks" with my mouse every time I need a bulleted list or image.

The new editor in WordPress is no doubt an attempt to win users from the Squarespaces and Wix.coms of today's market. I wonder, however, if this comes at the cost of alienating technical users or users who are simply used to the old look and feel. Giant changes are always likely to have push back, especially with a user base as large as that of WordPress. Given, however, that the Classic Editor plugin already has over one million users, I'd say that this change has a bigger negative opinion than the WordPress powers-that-be might be willing to admit. It will be interesting to see how things progress over the next few months. I'm just thankful that the Classic Editor plugin even exists.

Last week, while on vacation in the northwestern corner of the United States, I was notified by a few people that my article Stack Overflow Hates New Users had made it onto a news site I was previously unfamiliar with, hackernoon (actually, the article was a part of a longer piece entitled The decline of Stack Overflow). It's rather amusing that an article I wrote over four years ago is still being cited by people, and I think that says a lot about the community surrounding the venerable question and answer site. I've been a user at said site for over four and a half years, and I have a few additional things I'd like to discuss.

Seeing as I have already written about things I learned using Stack Overflow, a list of my gripes with Stack Overflow, and a few suggested improvements for Stack Overflow, I won't rehash those here. What I would like to talk about, however, is how I think new users should approach the Stack Overflow experience:

Search, Search, and Search Again

I cannot stress this point enough. Before you ask a question on the site, search it to see if a similar question has already been asked. Over 95% of the time, at least one person will have already asked a question that should point you in the right direction. Don't use the built-in search, by the way; like many site search boxes, it's pretty lousy. Search with Google using the site:stackoverflow.com operator, and include any relevant keywords that could narrow your search (searching is an art form and non-trivial!).

Ask Detailed Questions

This is probably the biggest mistake new users make: they don't provide enough detail. Asking detailed questions takes some practice to master (I still haven't mastered this). Before you click that "Ask Question" button, ask yourself these questions:

  1. Is my question clear, concise, and easy to read?
  2. Have I mentioned what I've tried in the question I'm about to post?
  3. Have I provided all relevant details about my question (any specific library versions, or specific database flavors, or the operating system in use, etc.)?
  4. Do I have an idea of the direction to head, or am I flying blind? Have I made it known in my question that I know (or don't know) where to go?
  5. Where else have I looked for solutions? Did I mention these in my question?

Being short and to the point, yet detailed, is a difficult balance to achieve, but find that balance and you'll reap the rewards.

Be Patient

It's very difficult to get a lot of rep points in the beginning, so be patient; building trust takes time. Set mini-goals for yourself. I personally set a goal to get to 2000 rep points so I could edit other posts without having to have my edits approved. Once I reached that goal, I stopped worrying about points altogether (it's not worth worrying about)! Also, be courteous to everyone, even the jerks on the site (of which there are a number). Know-it-alls tend to like to flaunt their intelligence, and Stack Overflow is an outlet through which they can scratch that itch.

Join a Sister Site

I'm a big fan of Arqade, a sister site to Stack Overflow focused on video games. The community there is much more friendly and welcoming, and if you earn enough reputation at a site like that (only 200 points), you'll automatically get 100 bonus points on every other Stack Exchange site!

Stack Overflow is a great resource to use, but it's one that I keep at arm's length. These days, I tend to ask more questions than I answer, and I often find answers to questions I have through other people's postings. It's easy to get swept up in the competitiveness of the site at the beginning, but if you avoid doing that, you'll have a much more pleasant experience.

Migrating to HTML 5

Jan 20, 2015

After sitting on an XHTML Strict template for years and years, I've finally migrated this site's theme to HTML 5. A number of new elements have been put to use, styles have been trimmed a little, and I'm using one less web-font. Hopefully I haven't broken too much; if you spot something, let me know in the comments below. As always, expect sporadic updates as I add polish.

I'm currently working on converting this site's layout to a fluid, responsive-design variant, which will make the site much more usable on mobile devices (as of this writing, the mobile experience is pretty bad). In my efforts to improve the design, I noted that the browser on my Android phone didn't respond to my changes at all. The Responsive Design View in Firefox yielded acceptable results, as did the developer tools available in Chrome (which, by the way, are pretty terrible). So why did my phone look so different?

The answer has to do with the concept of the viewport. I found a terrific pair of articles detailing the ideas behind viewports on both the desktop and mobile devices (be sure to read the desktop article first; it sets the stage for the second). It turns out that my page's header was missing a key meta element, whose very existence we can thank Apple (of all people) for. The tag looks like this:

<meta name="viewport"
      content="width=device-width, initial-scale=1, maximum-scale=8" />

Using this meta entry allows devices to scale their layout viewport appropriately. Adding this one element fixed my phone, causing the site to render as expected. There are other articles that discuss this element better than I could, so give them a look. And don't forget this one line if you're designing responsively; it'll save you a lot of potential headache!

Even though the site aggravated me at first, I still occasionally troll Stack Overflow. One of the leading problems I see in questions pertaining to PHP & MySQL, is people's use of the MySQL extension in PHP. This extension, it turns out, is being deprecated. But does the documentation reflect this fact? Yes and no.

Certain function pages, such as mysql_real_escape_string, have big red boxes at the top indicating that the extension is being deprecated. "Don't use this", they seem to shout. Other function pages, however, such as the mysql_result page, don't have these warnings. Likewise, the top-level MySQL Drivers and Plugins page lists the MySQL extension first, with no indication whatsoever that the extension is being deprecated.

At the very least, every single documentation page that deals with the MySQL extension in any form or fashion, needs to include information about its intended deprecation. Otherwise, thousands upon thousands of programmers will write code using a plugin that is quickly nearing it's end-of-life. Which, based on what I see at Stack Overflow, already seems to be the case.

My last post dealt with a few issues I encountered during my first 30 days of using Stack Overflow. I would now like to propose a few ways I think some of these can be smoothed over. This will be my final post on the Stack Overflow topic; I think I've pretty well worn it out.

Hide user scores by default
New users shouldn't see other people's reputation scores by default. If the primary goal of scores is to identify the most "helpful" users in the community, use a color-coded or sliding scale graphic instead. Perhaps low-scoring users are on the "cool" end of the spectrum, while high-scoring users are on the "hot" side. Hiding scores by default seems to me to be an easy way to take away some of the competitiveness, while leaving some sort of means for determining a contributor's "worth" to the community.
Implement a back-off for power-users
The more "powerful" a user (i.e. the higher their score), the longer they should be locked out from answering new questions. A delay of this sort doesn't need to be giant; a few minutes would suffice. For example, perhaps new users can answer questions immediately, an "average" user can't answer new questions for three minutes, and power users are locked out for five minutes. This would encourage newer users to answer the newest questions, and would make it harder for power users to snatch up points for simply being the fastest on the block.
Implement a delay for accepting answers
If a back-off delay is implemented, clearly an answer acceptance delay is needed. Let's say that answers for new questions cannot be accepted for 10 or 15 minutes after they are asked. This would give everyone a chance to form well thought out answers to the questions asked, and would help keep new users from gaming the back-off delay.
Create a better user manual
Hire a competent writer and create a better user manual. This should be self explanatory.

If you have thoughts on these suggestions, I'd enjoy hearing them.

Having used Stack Overflow for over 30 days now, I'd like to share a few of my personal gripes with the site. Those of you who are frequent Stack Overflow contributors may disagree with the views I express below; if so, feel free to leave a comment. I'm going to try my hardest to point out issues that seem particularly problematic to new users. My next post on this subject will cover a few ways I think some of these problems could be smoothed over.

Stack Overflow is a game
This is a controversial viewpoint, but one that has some merit, especially for the dedicated user base. Every Stack Exchange website has both points and achievements. Given those two incentives, many dedicated users seem to play solely to "win." If you look at the activity reports for some of the top-scoring users, you'll find that they spend many hours a day, every day, answering questions and leaving comments. I recently viewed one person's profile, and saw that he had answered several questions an hour for over 8 hours in a row (he must not have a very demanding job or life)! How can new users "compete" with folks like this?
The game is, by design, not winnable
The ultimate irony is that the "game" cannot be won. There's no end-point, no game-over screen; just a never-ending leader board. Other than seeing your name among the top contributors, and apart from gaining a small set of user privileges as your score increases, what's the point? Why have points and achievements at all? I can understand wanting to foster constructive discussion, but won't that naturally occur given the question and answer voting mechanisms? It seems to me that if points were left out altogether, good answers and good questions would still be easy to find, taking precedence over less useful material.
Being fast is often a requirement
This is my pet peeve with the site. Because Stack Overflow itself has so many "players," and because they are all playing for the same points, it's inevitable that faster players end up scoring more than slower ones. I pity the person who yearns to contribute, but whose motor impairment makes it difficult to respond in a timely manner. So too the person who may really know what they're talking about but, because they post in haste, their answer contains a mistake, resulting in its being down-voted by trigger happy power users.
The instruction manual stinks
Like most documentation developed by engineers, the training manual is way too thin. I cannot stress this highly enough. For the first few days on the site, I was overwhelmed by all the various rules and regulations. Most of my "education" came through mistakes I made early on, some of which resulted in a punishment to my score. It doesn't help that one must dig down into another site to find a fuller set of instructions, many of which should have been in the training manual to begin with. A set of tutorials describing how the site works is sorely needed. I'm a reasonably intelligent person, and if I'm having trouble, think about how many others are out there having the same problem.
Established users are generally rude to newbies
This is a problem that cannot be solved. People are jerks. But the points system really exacerbates the issue. I've seen so many negative comments towards new users trying to answer questions; especially towards those who make mistakes in their answers. I had hoped that people would be friendlier, seeing as everyone's goal is (hopefully) to learn and grow as programmers. Sadly, that wasn't the case. I guess I should have known better.

Now that my 30 days are over, I will be dialing back my use of the site considerably. My original intention of helping others may have been noble, but it turns out that there are already too many operators standing by. I question whether that game is really worth playing.

In my last post, I complained about my initial experience with Stack Overflow. I decided to give myself 30 days with the service, to see whether or not I warmed up to it. Now that those 30 days are over, I will be posting several of my thoughts and observations. This first post won't be about the site itself; instead, it will cover some of the things I learned during my 30 days. A second upcoming post will cover some problems I think exist with the Stack Overflow model, and my final post will provide a few suggestions for how I think things can be improved.

Let me first say that I learned a lot simply by browsing the site. Reading existing questions and their answers was fascinating, at least for the programming topics I care about. Some of what I learned came through mistakes I made attempting to answer open questions. Other bits of information just came through searching the web for the solution to someone's problem (something that a lot of people at Stack Overflow are apparently too lazy to do). Without further ado, here's a list of stuff I learned, in no particular order (each item lists the corresponding language):

C (with GNU Extension), PHP (5.3+)
The true clause in a ternary compare operation can be omitted. In this case, the first operand (the test) will be returned if true. This is a bizarre shortcut, and one I would never personally use. Here's a PHP example (note that there's no space between the question mark and the colon; in C, a space is necessary):
$a = $b ?: $c; // No true clause (too lazy to type it, I guess)
$a = $b ? $b : $c; // The above is equivalent to this
Regular Expressions (Perl, PHP, possibly others)
The $ in a regular expression doesn't literally match the absolute end of the string; it can also match a new-line character that is the last character in the string. Pattern modifiers are usually available to modify this behavior. This fact was a surprise to me; I've had it wrong all these years!
Bash
I found a terrific article that details the differences between test, [, and [[.
Firefox Extensions (XUL, JS)
You can use the addTab method in the global browser object to inject POST data to a newly opened tab.
Perl
The way I learned to open files for output in Perl (over a decade ago) is now not advised. It's going to take a lot of effort on my part to change to the new style; old habits, and all that.
# Old way of doing it (how I learned)
open OUT, "> myfile.txt" or die "Failed to open: $!";

# The newer, recommended way (as of Perl 5.6)
open my $out, '>', "myfile.txt" or die "Failed to open: $!";