Category: Web Development

With so many video codecs and containers, it’s easy to get confused. Here’s a simple explanation.

Codec (Video Format)

Codec stands for coder-decoder. Video codecs are algorithms for encoding and decoding video data.

• An encoder compresses video streams, which reduces the amount of data for storage and transmission.
• A decoder reverses the conversion for playing or editing a video stream.

For simplicity, you can think of a video codec as the video format.

Examples of video codecs are H.261, H.263, VC-1, MPEG-1, MPEG-2, MPEG-4, AVS1, AVS2, AVS3, VP8, VP9, AV1, AVC/H.264, HEVC/H.265, VVC/H.266, EVC, LCEVC

Currently, the most popular codec is AVC/H.264.

Container (File Format)

With respect to video, a container is a data storage. It can include compressed video and audio sequences, subtitles, service information and metadata. It is a package or bundle.

For simplicity, you can think of a media container as the file format.

Examples of media containers are MPEG-1 System Stream, MPEG-2 Program Stream, MPEG-2 Transport Stream, MP4, MOV, MKV, WebM, AVI, FLV, IVF, MXF, HEIC

Currently, the most popular codec is MP4.

There are many ways and tools you can enhance a photo. If you’re a professional photographer, then you’ll likely have advanced methods, but for the average person, you’ll probably just want some quick and easy solutions. Like most average people, my photos are mostly taken from my phone (currently, Google Pixel 8). However, when vacationing, I also take a lot of video using my Insta360 X3 camera, and I’ll occasionally want to take snapshots of a video frame to add to my photo collection. With this in mind, here’s my current (simple) workflow for upscaling and enhancing photos.

Enhance a Photo Using Google Photos

First, upload your photo to Google Photos. Then, use one of the presets to enhance the photo. Here’s an example photo without any enhancements applied.

The average person might that the photo looks fine, but it can significantly be improved. Here’s how the photo looks when you click on each of the suggested improvement options.

Note that Color Pop tried to isolate the subject and convert everything else to grayscale. It’s not perfect because the subject’s right arm is partially gray. To fix this, you could select the subject in Photoshop either manually or automatically, invert the selection, and convert the selection to grayscale.

Now, let’s say that you have a heavily underexposed (dark) photo. (Learn more about under and overexposed photos).

If you click the “Enhance” or “Dynamic” options, you’ll get this.

For comparison, here are the photo’s input levels in Photoshop.

If I were to manually correct the exposure in Photoshop, this is what I’d get.

The photo is significantly improved, but it doesn’t look exactly like it does using the Google Photos presets.

If the Google Photos presets don’t look good enough, you can make many adjustments in the Settings tab. In the example below, I started by choosing the “Dynamic” preset, and then in the Settings tab, I increased the brightness.

So, for the average person, using Google Photos to improve photos is easy and usually adequate.

Enhance a Photo Using Topaz Photo AI

Topaz Photo AI can do many things to a photo, including

• remove noise
• sharpen
• adjust lighting
• balance color
• recover faces
• preserve text
• upscale

You can also just run autopilot and let Topaz choose settings for you.

For me, I mainly use Topaz to enlarge (upscale) photos, remove noise, which can result from adjusting the levels of a heavily underexposed photo, and to sharpen photos. These improvements are particularly useful when I take a snapshot of a 1920×1080 video frame. For example, here’s a frame from a video.

I want to zoom in on the subject, crop it, enlarge it, and enhance it. Here’s the zoomed-in part cropped. The dimensions are 1048 x 589 px.

Now, I’ll drag it into Topaz and run autopilot to upscale and enhance the photo. It will take a minute to process. Here’s how the photo looks enlarged by 34% before enhancing it with Topaz.

Here’s how it looks with Topaz enhancements applied.

There is a difference, but it will be more obvious when you zoom in. Below is a comparison zoomed in at 67% before and after using Topaz.

At this point, you can copy the upscaled and sharpened photo from Topaz and paste it into Google Photos to enhance it.

Topaz Photo AI isn’t perfect, but depending on the original photo, it can often product amazing results.

Chances are you’ve taken photos where the faces are too dark. This can be due to a low-quality camera and/or incorrect camera settings. Fortunately, there is a way to easily fix this using Photoshop. As an example, I’ll use this photo of me in the Wynwood district of Miami (below). Notice how my face lacks detail because it’s underexposed (too dark).

Levels

First, I like to check and adjust the levels for the image. Go to Image > Adjustments > Levels. You’ll see a histogram like this

In the input levels, there are 3 markers from left to right

• black marker (left) = shadows
• gray marker (middle) = midtones
• white marker (right) = highlights

The left and right markers should touch the left and right edges of the histogram. In this case, there is a small gap between the right marker and the right edge of the histogram, meaning the image is a little underexposed. If we move the right marker to the right edge of the histogram, the image will become a little brighter.

In this particular example, this adjustment isn’t significant. But, if there’s a big gap in the histogram like this

then adjusting the levels will make a big improvement to your image.

Exposure

If adjusting the levels doesn’t brighten your subject’s face enough, you can adjust the exposure of just the face. First, make a circular selection around a face.

Then, add a feather to it. Select > Modify > Feather. For a 1920 x 1080 image, I create a 50-pixel feather.

Then, adjust the exposure by going to Adjustments > Exposure

You will see a slider for Exposure. Drag it to the right to increase the exposure.

Notice how the face is brighter.

If I increase the exposure too much, the face will look nice and bright, but it won’t look natural against the darker surroundings.

The problem is there appears to be a white glow around the face. In this case, I can select just the subject or the subject’s face to limit the exposure adjustment area. In this case, no feather is needed.

Now, only the subject is brightened.

I can also tweak the levels by adjusting the shadows and the midtones so that the subject doesn’t look unnaturally bright against a darker background.

Before and After

As you can see in the before and after image below, my face looks much brighter while still appearing natural.

Many people wonder why their website doesn’t rank high for a particular keyword. Let’s take cybersecurity as an example because I am familiar with that space. A cybersecurity company usually has products and events, and a need to market them. One type of product is a tool to manage vulnerabilities. The industry refers to the topic as “vulnerability management” or “VM”. Marketers would then often ask, “why isn’t my VM product page showing up at the top of Google when people search for “vulnerability management”. Likewise, if the company has an annual user conference with session talks about cybersecurity, marketers will again often ask why their conference website doesn’t show up for the keyword “cybersecurity”. The main reason lies in search intent.

Search Intent

When people search for “vulnerability management” or “cybersecurity”, they are most likely not searching for a VM product or a cybersecurity conference. These generic terms likely indicate that they are searching for information about them rather than a tool or a conference. Consequently, search results for those terms show informational pages containing content explaining what they are as if a user searched for “what is vulnerability management” or “what is cybersecurity”.

There are 4 types of search intent:

• Navigational intent: Users want to find a specific page (e.g., “reddit login”)
• Informational intent: Users want to learn more about something (e.g., “what is seo”)
• Commercial intent: Users want to do research before making a purchase decision (e.g., “best coffee maker”)
• Transactional intent: Users want to complete a specific action, usually a purchase (e.g., “buy subaru forester”)

Therefore, for the VM tool and cybersecurity conference examples above, the correct search terms to check would be something like

• (best) vulnerability management tool
• cybersecurity conference

The qualifiers “tool” and “conference” make it clear what people’s search intent is so they find content relevant to what they are actually looking for. But how do you know what keywords people are searching for? This is where keyword research tools come in.

Keyword Research

To do keyword research, you can use a free tool like Google Keyword Planner or a paid tool like Semrush. I’ll use Google Keyword Planner. You can use it to

• discover new keywords and
• get keyword search volume and forecasts.

Continuing with our example, let’s see how many people search for the following keywords:

• vulnerability management
• vulnerability management tool
• cybersecurity
• cybersecurity conference

I enter those keywords as shown below…

and Google spits out the following data

The keyword “vulnerability management” gets many more searches per month compared to “vulnerability management tool”. But, since people searching for “vulnerability management” are probably not looking for a tool or software, people will likely ignore your VM product page even if it does show up in the #1 spot. The keyword “vulnerability management tool” gets fewer searches, but ranking for it will at least match your product page with a targeted group of interested people. Therefore, you’ll want to optimize your page for the keyword “vulnerability management tool”, e.g. by adding the word “tool” to your page copy, and monitor your Google page rank for that keyword.

As for the conference example, though the keyword “cybersecurity” gets a huge number of searches per month, it’s such a generic term that most people are probably not looking for a cybersecurity conference when they search for “cybersecurity”. On the contrary, the keyword “cybersecurity conference” is more specific / targeted and gets a decent search volume. Therefore, that is the term you should optimize for and monitor your page rank for.

In addition to the keywords you may already know about, you can using Google Keyword Planner to discover other relevant keywords. For example, if I enter “vulnerability management tool”…

I get the following results.

It looks like some companies like “Qualys” and “Rapid7” are known for having vulnerability management tools, so people looking specifically for their respective product pages just use the company name as a qualifier. However, there are some non-brand keywords that get a decent amount of searches as well, including “vulnerability management program” and “vulnerability management process”. While these keywords can be included in a VM product page, they may deserve to have their own pages.

How Long to Rank High

Now that you know what keywords to target, another misunderstanding people have is that adding keywords to a page will result in a higher page rank in just a matter of weeks. First of all, keywords are just one of many factors that affect page rank. SEO is a long-term strategy. Unless you’re looking to rank high for a long-tail keyword – a keyword so specific that there is very little competition – don’t expect immediate results.

Launch an EC2 Instance

• Log in to AWS and go to EC2 > Instances > Launch an Instance
• Enter a name. I’m calling mine “My Web Server”.
• For Application and OS Image, I’ll just choose the default, which is “Amazon Linux 2023”.
• For Amazon Machine Image (AMI), I’ll choose the default, which is “Amazon Linux 2023 AMI”
• Under Key Pair, click “Create new key pair”.

Amazon EC2 can easily create a key pair for you. Just enter a key pair name. I chose “aws-ec2”. OpenSSH is available on Linux, Mac, and Windows 10+, so keep the default key format of .pem. On Windows 10+, OpenSSH is an optional feature you must install. Click the “Create Key Pair” button. The private key will be downloaded to your computer. Keep it in a safe place. You will need it to SSH into your EC2 instance.

In the Network Settings section, since we want to SSH into the EC2 instance and we want to be able to browse our website over HTTP and HTTPS, check those checkboxes.

Leave everything else at their defaults. Review the summary and click the “Launch Instance” button.

You will then see your EC2 instance listed. Wait for the “Status check” to change to “2/2 checks passed”.

Once your instance has been set up, click the button to connect to the instance.

SSH into EC2 Instance

You have a few options to connect to the EC2 instance. For simplicity, choose EC2 Instance Connect. This will open a new browser tab with shell access. Leave the default username as “ec2-user”.

You’ll notice the command prompt changes to ec2-user@ip-172-31-47-114 which is my default username (ec2-user) followed by my EC2 instance’s private IP (ip-172-31-47-114).

Install Apache

Since we installed Amazon Linux 2023, follow these instructions to install Apache. Since we don’t need MySql and PHP, ignore the commands and instructions for those. For example, instead of

sudo dnf install -y httpd wget php-fpm php-mysqli php-json php php-devel

just do

sudo dnf install -y httpd

Make sure to follow the instructions to set the file permissions so that Apache can serve the website.

• The Amazon Linux Apache default document root is /var/www/html
• The Apache config is at /etc/httpd/conf/httpd.conf
• The Apache logs are in /var/log/httpd/

To view Apache errors, run the following command

sudo tail -100 /var/log/httpd/error_log 

Test that Apache works by going to the public IP address WITHOUT “https”, e.g. http://34.229.240.7/.

Set Up SSL/TLS

These instructions show how to create a self-signed certificate and a CA-signed certificate. For a self-signed cert, you don’t need a domain name. You can access your website over https by IP address, e.g. https://34.229.240.7/

For a CA-signed cert, you can follow these instructions to automate certificate renewals using Let’s Encrypt with Certbot. You can also use AWS Certificate Manager to manage and automatically renew certs.

Get a Fixed IP Address

The default IP address that AWS gives you is dynamic (will change whenever the server restarts). To get a static (fixed) IP address, get an Elastic IP Address. Once you get one, try to access your website over https, e.g. https://35.173.7.249/

Put Your Website in a GitHub Repo

1. Create your website locally in a folder. If you have an existing website under git version control with a lot of history and you want to remove the history, git clone the repo into a new folder, delete the “git” folder, and then run git init.
2. Create a new repo in GitHub
3. Push your local repo to GitHub

git remote add origin [email protected]:...
git push -u origin

Clone the GitHub Repo to Your EC2 Server

1. In your EC2 CloudShell (web-based terminal), install git (sudo dnf install git -y). Learn more.
2. Generate a new SSH key and add it to the ssh-agent. Since your EC2 instance is Linux, follow the instructions for Linux.
3. Add your new SSH key to your GitHub account
4. Clone your GitHub repo to your EC2 instance. I’m going to clone it to my home folder.
5. In GitHub, get the SSH URL of your repo.

Then, in your home folder, clone it

You will then see a new folder containing your website files from the GitHub repo.

Since my website document root is at /home/ec2-user/my-website/www, we need to update the Apache default document root (/var/www/html) to reference that path by editing the Apache config.

sudo nano /etc/httpd/conf/httpd.conf 

• Change all references of /var/www/html to /home/ec2-user/my-website/www 
• Change all references of /var/www to /home/ec2-user/my-website 
• Restart Apache (sudo systemctl restart httpd)

Update Folder Permissions

If you try to view your website, e.g. by going to https://35.173.7.249/, you will probably get a “Forbidden” error. To better understand this error, view the Apache error log.

sudo tail -100 /var/log/httpd/error_log 

You will probably see an error like this

[Sat Dec 23 01:36:55.545345 2023] [core:error] pid 89394:tid 89446 Permission denied: [client 135.125.246.189:49368] AH00035: access to / denied (filesystem path '/home/ec2-user/my-website') because search permissions are missing on a component of the path

To fix this, follow these instructions on how to update file permissions

But replace /var/www with your website document root. In my case, I changed it to /home/ec2-user/my-website

According to the Apache docs, you also need to change the file permissions of all parent folders. In my case, the following folders need to be executable

• /home
• /home/ec2-user
• /home/ec2-user/my-website

ls -la 
chmod +x . 
cd .. 
# repeat up to the root

When you’re done, you should be able to view your website in a browser.

Optional

• Install Node.js and npm on EC2
• Note that the dotenv package is no longer needed in Node.js v20.6.0+ because those Node versions have built-in support for .env. Learn more

If you’re a non-technical person who is part of a marketing team working for a company that depends a lot on a website, chances are you will often need to ask a team of web developers to make website updates for you. Your particular website may not be easily updated using a content management system (CMS), and even if it could, many non-technical people would rather just send an email to request their website changes. Asking developers to update a website is fine, but only if the update requests are clear. Otherwise, the requestors risk wasting their time and other people’s as well. Unfortunately, the reality is most people don’t know how to clearly communicate their change requests. There are many website annotation tools that claim to be able to simplify the communication process, but in real-world situations, I haven’t found any that were good enough. Plus, adding a new tool requires learning something new, which many people are unwilling to do or don’t have time for.

In this post, I’ll share one approach that non-technical people can use to easily and clearly communicate website change requests to minimize misunderstandings, delays, and lots of back-and-forth messages. And since most people already know and are comfortable using MS Word or Google Docs, this approach only requires a word processor.

Website Sections

Most websites contain a vertical series of sections. For example, Adobe has a product page for its video editing software, Adobe Premier. This web page can easily be divided into a bunch of sections.

Change Request Doc

Since a picture is worth a thousand words, it’ll be a lot easier to show a screenshot of a section of a web page rather than try to explain the section using words. And since you may want to move some sections around, it’s helpful to number each section. And since you may collaborate with other people in requesting website changes, we’ll use MS Word or Google Docs for our change requests. I’m going to use Google Docs because I find it easier to use.

1. Create a new Google doc
2. Give it a name like “Adobe Premier Product Page Changes”.
3. Change the page margins to 0.25″ on all sides
4. Under View, uncheck “Show print layout” if it is checked.
5. At the top of the doc, put the URL to the page, e.g. https://www.adobe.com/products/premiere.html
6. Insert a table containing 3 columns and 20 rows.
7. In row 1, cell 1, enter “#”
8. In row 1, cell 2, enter “SECTION”
9. In row 1, cell 3, enter “CHANGES”
10. In the first column, enter a consecutive number in each cell starting from 1 and make it narrow enough just for the numbers
11. Take a screenshot of each section and paste them in the middle column
12. In the right column, describe your change request.

Here’s an example.

View the sample Change Request Google Doc.

Many website files include PDFs. These PDF files are usually much larger than other file types and can take up a lot of space. You may want to keep all website files like images and PDFs (binary files) together with your HTML, CSS and JS files (text files) and put them all in version control, like GitHub, but there are downsides to this:

1. Git version control is designed for text files, not binary files. Even though you can use Git LFS so you can version your binary files, there are simpler, better alternatives.
2. Website images are better served from an image CDN like Cloudinary or ImageKit. These services will automatically and quickly optimize images on the fly.
3. PDF files are better served from a CDN. Amazon AWS S3 can be used to store your PDFs with versioning and AWS CloudFront can serve those PDFs from a CDN. With CloudFront, you can also write a function to redirect one PDF file to another in case you need to delete a file.

As of this writing, the AWS CloudFront CDN has 218 global edge locations.

The steps below describe how to set up AWS S3 and CloudFront to host PDFs and to set up redirects.

Note: you can create redirects using AWS Lambda functions (launched in 2017), but they are more complicated and cost 6 times as much as the cost of CloudFront functions (launched in 2021). Learn more.

1. Create an S3 bucket

Log in to the AWS console, go to S3, and click “Create bucket”. Choose a bucket name like “pdfs”.

Since you want people to be able to access the PDFs, uncheck “Block all public access” and check “I acknowledge that the current settings might result in this bucket and the objects within becoming public.”

If you want, click the radio button that enables versioning

Ignore the other options, if you want, and then click the “Create bucket” button.

2. Upload PDFs

You can drag and drop your PDFs to upload them. If you have many PDFs, like thousands, then it’s better to use the AWS CLI S3 Sync command.

As a test, I just uploaded 2 PDF files/

3. Create a CloudFront Distribution

In the AWs console, go to CloudFront and click “Create Distribution”. For “Origin domain, choose the Amazon S3 bucket you created in step 1.

For the viewer protocol policy, choose “Redirect HTTP to HTTPS” since that’s a good policy IMO.

Ignore all other options, if you want, and click the “Create Distribution” button.

Now, the PDF files in your S3 bucket will be available in a CDN at the CloudFront domain provided, e.g. d2a5k3j4u1zr32.cloudfront.net/test-pdf-1.pdf

4. Create a CloudFront Function to Redirect Requests

Click on the distribution and then click on “Functions” in the left sidebar.

Click the “Create Function” button and enter a name for the function, e.g. “Redirects”.

You will see 3 tabs: Build, Test, and Publish.

In the “Build” tab, enter the code below and customize as needed.

Note that there is a 10 KB limit on the size of your CloudFront function.

Click the “Save Changes” button and then click the “Test” tab. You will see a field labeled “URL Path” with a default value of “/index.html”.

Since we don’t have a redirect rule for that URL path, we don’t expect any redirection to happen. Click the “Test Function” button. You will see output like below indicated that the response URI is “/index.html” as expected.

Now, change the URL path to one you have a redirect for. In my example code, I am redirecting “/test-pdf-2.pdf” to “https://www.google.com”. Click the “Test Function” button. The output shows “https://www.google.com”.

Now, publish the CloudFront function. Click the “Publish” tab, then click the “Publish Function” button.

Click “Add Association” to associate the function to your distribution. Choose your distribution in the Distribution field. Leave Event Type as “Viewer Request” and ignore Cache behavior. Click the “Add association” button.

Note that you can only have one CloudFront function for a given cache behavior and event type.

Wait for the function to be deployed. Go back to the function list page and check the status column. It will say “Updating” for a few minutes.

Wait a few minutes. Reload the page. The status should change to “Deployed”.

Now, test out the redirect in production by going to the CloudFront URL of a path you have a redirect for. You should see the redirect work.

Using Lambda Functions

Make sure the location is set to us-east-1.

Go to the Lambda page and click “Create function”.

1. Enter a name for your function.
2. Under “Execution Role”, choose “Create a new role from AWS policy templates”
3. Enter a role name
4. Under “Policy Templates”, choose “Basic Lambda@Edge permissions (for CloudFront trigger)”. This is IMPORTANT. Do NOT choose “Create a new role with basic Lambda permissions”.

In the “Code” tab, enter the redirect code below and then click File > Save.

In order to test your code, you must deploy it first. Click the Deploy button.

1. Test your code by clicking the “Test” tab
2. Choose “Create new event”
3. Enter a name for the test
4. Replace the event JSON with relevant test data, e.g.

{
  "Records": [
    {
      "cf": {
        "config": {
          "distributionId": "EXAMPLE"
        },
        "request": {
          "uri": "/docs/test-pdf-2.pdf"
        },
        "response": {
          "status": "302",
          "statusDescription": "Found",
          "headers": {
            "location": [
              {
                "key": "Location",
                "value": "https://example.com"
              }
            ]
          }
        }
      }
    }
  ]
}

Click the “Save” button and then the “Test” button.

You will either see an error or a success response similar to what’s shown below.

Under “Actions”, click “Deploy to Lambda@Edge”. This will deploy the Lambda function to the CloudFront edge network.

1. Choose your CloudFront distribution from the dropdown list.
2. For CloudFront event, choose “Origin Response”.

The green banner will state that the function is being replicated, but that it will take a few minutes to complete.

Go to the CloudFront distribution. You’ll see the status “Deploying”. Wait till it changes to a date/time indicating the deployment has completed.

Invalidate the CloudFront cache for all objects using /*

When the trigger is created, it will create a new Lambda function version. Click on the “Versions” tab and then click the version number to see that the trigger is saved in the version.

You will then see the CloudFront trigger in the diagram and other saved details.

Test the redirect using the cURL command.

If you need to remove a Lambda function from a Cloudfront distribution,

1. go to the distribution
2. click “Behaviors”
3. choose a behavior and click “Edit”
4. Scroll down to “Function Association” and select “No association” for the function type
5. Click “Save changes”
6. Invalidate the Cloudfront distribution using /*

Put Redirect Data in an External JSON File

The instructions above work, but whenever you want to update the redirects, you have to edit the lambda JavaScript function and redeploy it to the Cloudfront edge. The deployment process takes about 5 minutes. To improve this process, we can move the redirect data to a JSON file in an S3 bucket. Then, you can just upload an updated JSON file, overwriting the existing file, and the updated redirects will work immediately. Here’s how to do that.

Create a JSON file containing all redirects like the following and upload it to S3.

{
  "/file1.pdf": { "to": "/file2.pdf", "statusCode": 301 },
  "/resources/file3.pdf": { "to": "/resources/file4.pdf", "statusCode": 301 },
  ... etc. ...
}

Add permissions to the lambda function to have read access to S3. Go to Lambda > Functions > and click on the function name. Then, go to Configuration > Permissions > Execution Role > and click on the role name.

A new tab containing the role’s permission will open. Under “Permissions policies”, click on the policy name.

That will open a new tab showing the permissions defined in the policy. Click the Edit button.

A new table will open showing the existing permission. Add the following S3 permissions. Replace “mybucket” with the name of the S3 bucket where you put the JSON redirect file.

"Effect": "Allow",
	"Action": [
		"s3:Get*",
		"s3:List*"
	],
	"Resource": [
		"arn:aws:s3:::mybucket",
		"arn:aws:s3:::mybucket/*"
]

The entire JSON object may look like this.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"logs:CreateLogGroup",
				"logs:CreateLogStream",
				"logs:PutLogEvents"
			],
			"Resource": [
				"arn:aws:logs:*:*:*"
			]
		},
		{
			"Effect": "Allow",
			"Action": [
				"s3:Get*",
				"s3:List*"
			],
			"Resource": [
				"arn:aws:s3:::mybucket",
				"arn:aws:s3:::mybucket/*"
			]
		}
	]
}

Back at the lambda function page, click on the Code tab and update the code as follows:

Replace the first 3 constant values with your S3 bucket’s region, bucket name and the path to the redirect JSON file.

const REDIRECTS_DATA_REGION = 'us-east-1';
const REDIRECTS_DATA_BUCKET = 'pdfs';
const REDIRECTS_DATA_OBJECT = 'docs/pdf-redirects.json';

Click “Deploy” so you can test the lambda function. Once you verify it is working, go to Actions > Deploy to Lambda Edge. Follow the remaining steps as shown above.

Resources

• https://making.close.com/posts/redirects-using-cloudfront-lambda-edge