- Table of Contents
- .NET Book Recommendations
- Getting Started with .NET
- The Microsoft .NET Framework
- The Common Language Runtime (CLR), the Common Type System (CTS), and the Common Language Specification (CLS)
- .NET Framework Class Library
- Visual Studio .NET
- .NET Enterprise Servers and .NET My Services
- .NET Compliant Languages
- C#
- Visual Basic .NET (VB .NET)
- ASP.NET
- XML Web Services
- ADO.NET
- XML.NET
- Windows Forms
- Why .NET?
- Displaying Errors with the Error Provider
- COM Interoperability
- Comparing Java and .NET
- Calling Unmanaged Code
- .NET Application Security
- Code Access Security
- .NET Standards Support
- Numeric Types in the .NET Framework
- Working with Strings
- Formatting Strings
- Trimming Character Strings
- Comparing Strings in .NET 2.0
- Arrays and Collections
- Arrays as Class Members
- Sorting a Multi-Dimensional Array
- Sorting a Multi-Dimensional Array with LINQ
- File I/O (System.IO)
- Working with File Names
- Using the File System
- Working with Files and Directories
- Monitoring the File System
- Working with Streams
- Working with Text Encodings
- Working with Date and Time
- Extending the DateTime Class
- Using DateTimeOffset
- Fun with Dates
- Exceptions
- Delegates
- Events
- Asynchronous Programming
- Asynchronous File I/O
- Timers
- Random Numbers
- Cryptographically Secure Random Numbers
- Serialization
- MultiThreading (System.Threading)
- Multi-Threading Overview
- The Managed Thread Pool
- Managed Threading
- Thread Synchronization
- Synchronizing Data Access
- Trace Debugging
- Tracing in .NET 2.0
- ASP.NET Trace
- Validating User Input in ASP.NET Web Pages
- Event Logging
- Monitoring Application Performance
- Accessing the Registry
- Accessing Environment Information
- Environment Variables in .NET 2.0
- Managing Windows Forms Applications
- Working with Email
- Working with Graphics
- Animating a Background
- Working with Images
- Drawing Cycloid Curves
- Simulating the Spirograph
- Building International Web Applications
- .NET Compact Framework
- Mobile Web Development with ASP.NET
- Speech Technologies
- Microsoft MapPoint Web Service
- Working with Typed DataSets
- Using Relationships in DataSets
- DataColumn Expressions
- Playing Simple Sounds
- Playing Sounds with .NET 2.0
- Returning an Image in a Web Page
- RSS
- Best Practices Project Structure
- Best Practices Application Blocks
- The Data Access Application Block
- The Exception Management Application Block
- Best Practices — Performance
- Best Practices — Performance and Scalability
- Best Practices - Testing
- Reading the Tea Leaves, 2005
- Predictions: A Look Back at 2005, and a Look Ahead to 2006
- .NET Downloads
- Application Deployment Overview
- Application Deployment — Versioning
- Application Deployment — Version Policy
- Application Deployment — Packaging and Distribution
- .NET Remoting Overview
- A Remoting Demonstration
- Remoting Configuration
- Remoting: Lifetimes and Leases
- Remoting: Other Issues
- Attributes
- Writing Custom Attributes
- Accessing Attributes in Code
- Reflection
- Class Design: Inheritance, Interface, or Composition?
- The TriTryst Game
- Console Applications in .NET 2.0
- New File I/O Methods in .NET 2.0
- Building Projects with MSBuild
- Unmanaged Callbacks in .NET 2.0
- Timer Troubles
- Non-Rectangular Windows Forms
- Windows Forms Transparency
- 10 Things I Hate About Visual Basic
- 10 Things I Hate About C#
- Background Processing with Idle Time
- Scaling Windows Forms
- Reading and Writing Binary Data
- New Memory Management Functions in .NET 2.0
- Compatibility Between .NET 1.1 and .NET 2.0
- Managed Debugging Assistants in .NET 2.0
- XDir: A Program for Viewing Directory Sizes
- The Microsoft.VisualBasic Namespace
- Operator Overloading
- Working with GPS Data
- Hidden Visual Studio Tools
- .NET 3.0
- The .NET 2.0 Stopwatch Class
- Nullable Types
- Drawing Rotated Text
- Unsafe Code
- Other .NET Languages
- Compiler Directives
- Safe Handles
- Predictions, 2007 Edition
- New Features in C# 3.0
- Generics
- Network Client Programming
- On the Misuse of Exceptions
- Maximum Object Size in .NET
- More on Maximum Object Sizes
- Keyed Collection Memory Limitations
- Matching String Endings
- Allocating Small Data Structures
- Grumbling About Limitations
- Some Thoughts on the Nature of What We Do
- Working with Predicates in Collections
- Working with DataReaders
- Outputting XML with XmlWriter
- Writing XML Data
- Working with Compression
- Another Look at Compressed Streams
- Compressing a Very Large File
- Canonical URIs
- Constructing URIs
- Using OneWayAttribute for Remote Calls
- Selecting a Garbage Collector
- Linked List
- Linked List Application - The MRU List
- Auto-implemented Properties in C#
- The HashSet Collection
- Looking Ahead: 2018
- An Experiment in Optimization
- A Larger Integer
- Extension Methods
- Language Integrated Query (LINQ)
- Variable Length Parameter Lists
- The ReaderWriterLockSlim Synchronization Primitive
- Sorting a Text File
- Sorting a Large Text File
- Using ListView with Large Data Sets
- LINQ One-Liners
- Regular Expression Optimization
- Random File I/O
- Computing the Size of a Structure
- More on Computing Structure Sizes
- UnmanagedMemoryStream
- Dynamically Loading Code
- Building a String Table
- Delegates Versus Function Pointers
- Visual Studio Editor Features
- A Simple Profile Timer
- New Features in C# 4.0
- IEnumerator or IList?
- New Features in .NET 4.0
- Set Operations with IEnumerable and HashSet
- Using File Locks
- Extending Object Functionality
- Clearing a HashSet
- When Hash Codes Matter
- Parsing Command Line Options
- Creating a Single-Instance Program
- Asynchronous Windows Forms Events
- The BackgroundWorker Component
- Fixing a Dumb Mistake
- Thinking About Multi-Threaded Programs
- JavaScript Object Notation
- Better JSON Processing with JSON.Net
- Useful .NET-related Sites
- Markov Models
- Building an Order 0 Markov Model
- Higher Order Markov Models
- Webmaster's Guide to robots.txt
- An Overview of the Parallel Extensions to .NET
- Parallel Extensions Synchronization Objects
- Thread Safe Collections
- A Bug and a Conundrum
- Another Bug and an Answer
- Task Parallel Library
- Good and Bad Ideas in C#
- Parallel LINQ
- Copying Large Files
- Replacing File.Copy
- Learning from Our Mistakes
- Symbolic Links
- There Is No Easy Fix
- Tracking Hurricanes
- Examining Hurricane Data
- Searching for Multiple Strings
- Simple JSON Processing
- Aho-Corasick String Searching
- Writing a Web Crawler
- Web Crawler Politeness
- Source Control Management
- Subversion
- Communicating with Datagrams
- Fun with Actions and Funcs
- The Future of Media
- The Importance of Metadata
- Of Comparison and IComparer
- IComparer, Comparer, IComparable, Oh My!
- Comparing Generic Types
- A Simple HTTP Server
- Quantizing DateTime Fields
- More Fun with the Garbage Collector
- Refactor, Don't Rewrite
- A Generic BinaryHeap Class
- A Generic File Sorter
- Birthdays, Random Numbers, and Hash Keys
- Random Selection from Large Groups
- Command Line Tools for Windows
- Reading and Writing, Bit by Bit
- Selecting the Top N Items from a Group
- Determining Website Content Encoding
- Benefits and Drawbacks of Syndication
- Pubsubhubbub
- Memory Use Misconceptions
- Risk, Lost Opportunity, and Other Hidden Upgrade Costs
- Culture Shock: from .NET to JavaScript
- Using .NET for a Startup
- Tracking Wikipedia Changes with IRC
- Browser Applications and the Same Origin Policy
- Handling the Unexpected
- Dealing with Growth
- Deleting the Oldest File
- Where Do I Put Stuff?
- .NET Timer Resolution
- Exploring Options for Better Timers
- Using the Windows Timer Queue API
- Locks Aren't Slow
- Alternatives to Locks
- Lock Free Concurrent Collections
- The BlockingCollection Class
- Customizing BlockingCollection
- What Time Is It? Daylight Saving Time and Computers
- Using enums to Save Memory
- New File Operations in .NET 4.0
- Building a Hierarchy of Rectangles
- A Faster File Copy
- Constants Are Forever
- The Dangers of Floating Point
- Goto is Not Inherently Evil
- The Weakest Link
- Reducing Memory Required for Strings
- Grouping with LINQ
- HttpListener "Gotchas"
- Extension Methods Are Evil
- Finding the Registered Domain in a URL
- Drawing Text
- Obfuscating Sequential Keys
- Properties of Obfuscated Keys
- Finding Changes Between Two Lists
- Using the ConcurrentBag Collection
- Never Sleep!
- Shuffling and Sorting
- Viewing Large Text Files
- Use the Right Tool
- Why GetHashCode Matters
- Optimization Guidelines
- Timer Differences
- The Mutex
- Modifying a Working System
- Building a New Type of Stream
- More Large File Problems
- A Better File.Copy Replacement
- Throwing the Wrong Exception
- Approximate Counters
- Monitoring a Timer
- Combining Consoles and Forms
- Embedding a Text Resource
- Handling Concurrent Downloads
- The Importance of Domain Knowledge
- Stupid Programmer Tricks
- Aho-Corasick Revisited
- Expressiveness is the Soul of Brevity
- Fun with Anonymous Types
- Simplifying a Multi-Threaded Application
- Work Smarter
- The Skip List Data Structure
- A More Memory-Efficient Skip List
- Selection Revisited
- Why Async?
- What the Future Holds
- The "Roslyn" CTP
- Where We've Been
- Informit Reference Library
Obfuscating Sequential Keys
Last updated Mar 4, 2011.
Imagine that one of the features of the Web site you're creating is the ability to share an item: an article that somebody posted, a video, or some other content that's identified in your database by a sequential identifier. So the first content item in the database is 1, then 2, 3, etc.
It's trivial to create share URLs that that allow users to access the items. That is, your URLs could be of this form:
http://www.example.com/29
There are two problems with such a scheme. First, you're susceptible to somebody trying to scrape your entire site by writing a program that creates sequential IDs. Perhaps more importantly, your keys are not all the same length. Your keys will grow in length as the number of items in your database increases. That is less friendly to automated tools than a fixed-length key.
One solution would be to encode the keys in hexadecimal, but that ends up giving keys with lots of leading zeros. A 64-bit hex representation of "29", for example, would be 000000000000001D, which users would think looks funny and code that works with the keys would have to be sure to 0-pad things. Plus, it doesn't address the sequential problem.
A better solution is to obfuscate ("encrypt" is too strong a word) the key to make it more difficult for somebody to guess the sequence, and make sure that all keys are the same length.
We'll discuss those two items separately.
Making a fixed-length key
The easier of the two operations is to create a fixed-length key. As you saw above, hexadecimal encoding will do it, but the keys are longer than they need to be (16 characters) and all those repeated zeros are kind of disappointing. A more compact representation that gets rid of leading zeros is Base64 encoding. That encoding uses the characters A-Z, a-z, +, and / to encode binary data. A 64-byte key using this encoding is 12 bytes long, and the last character can be removed because it's always '=' (to indicate "filler").
So, using base64 encoding, the 64-bit value 1 will map to AQAAAAAAAAA. 2 maps to AgAAAAAAAAA. Obviously, that doesn't solve the sequential problem and we've traded leading zeros with trailing A's, but it gives us a way to generate fixed-length keys that are shorter than the 16-byte hexadecimal keys.
The .NET Framework has a method, Convert.ToBase64String, that will return a base64-encoded string from a byte array. And we can turn get the byte representation of a UInt64 by calling BitConverter.GetBytes. So turning a 64-bit number into a base64-encoded string is very easy:
public static string KeyToString(ulong val)
{
ulong encodedKey = EncodeUInt64(val);
// get bytes
byte[] keyBytes = BitConverter.GetBytes(encodedKey);
// get base64 value
string keyString = Convert.ToBase64String(keyBytes);
return keyString;
}
That turns out to be less than ideal for two reasons. First, every key generated will have a trailing '=' character to indicate padding. That's there because each base64 character represents 6 bits, and bytes is 64 bits--which isn't divisible by 6. The padding says, in effect, "assume two zero bits." But since we know that all of our keys represent 64-bit values, we can just strip the trailing '=' from the generated key.
The other problem is that base64 encoding uses the characters '+' and '/', which have special meaning in URLs. Since we're going to use the generated keys in URLs, we have to map those two characters to something else. The modified base64 for URL applications maps '+' to '-', and '/' to '_'.
With those changes, our KeyToString method becomes:
public static string KeyToString(ulong val)
{
// get bytes
byte[] keyBytes = BitConverter.GetBytes(val);
// get base64 value
string keyString = Convert.ToBase64String(keyBytes);
// strip trailing =
keyString = keyString.Substring(0, keyString.Length-1);
// convert + to - (dash) and / to _ (underscore)
keyString = keyString.Replace('+', '-');
keyString = keyString.Replace('/', '_');
return keyString;
}
Converting a key string back to a number is just the reverse:
- map '-' to '+'
- map '_' to '/'
- add the trailing equal sign
- call Convert.FromBase64String to get a byte array from the key
- call BitConverter.ToUInt64 to get the key value
Here's the code.
public static ulong StringToKey(string keyString)
{
// convert - to +, and _ to /
keyString = keyString.Replace('-', '+');
keyString = keyString.Replace('_', '/');
// add the trailing =
keyString += '=';
// convert to bytes
byte[] keyBytes = Convert.FromBase64String(keyString);
// and get the key
ulong key = BitConverter.ToUInt64(keyBytes, 0);
return key;
}
We can test that with a simple bit of code.
static void DoKeyTest()
{
for (; ; )
{
Console.WriteLine("Enter value to be encoded. Blank line to exit.");
Console.Write(">");
string sval = Console.ReadLine();
if (string.IsNullOrWhiteSpace(sval))
break;
ulong val;
if (ulong.TryParse(sval, out val))
{
string keyString = KeyToString(val);
ulong key = StringToKey(keyString);
Console.WriteLine("{0:X16} -> {1} -> {2:X16}", val, keyString, key);
Console.WriteLine();
}
}
}
Here's the output from a few simple tests:
Enter value to be encoded. Blank line to exit. >1 0000000000000001 -> AQAAAAAAAAA -> 0000000000000001 Enter value to be encoded. Blank line to exit. >2 0000000000000002 -> AgAAAAAAAAA -> 0000000000000002 Enter value to be encoded. Blank line to exit. >1234567890123456 000462D53C8ABAC0 -> wLqKPNViBAA -> 000462D53C8ABAC0 Enter value to be encoded. Blank line to exit. >
Those keys probably look somewhat familiar if you've spent much time on YouTube. YouTube video keys are base64 encoded, but I know nothing about how they're generated. That is, they might really be random keys. Or they could be sequential keys that are obfuscated and encoded in much the way I describe below.
Now that we have code that will create a fixed-length key from a 64-bit number (and convert it back properly), let's look at solving the sequential key problem.
Obfuscating a number
When obfuscating a number, you have to make sure that two properties hold true:
- The mapping can be reversed.
- The mapping is unique.
If the 64-bit value 0x0000000000000001 maps to 0x3F3E3D3C3B3A3907, then you have to make sure that the reversal operation will map 0x3F3E3D3C3B3A3907 back to 0x0000000000000001.
Also, 0x0000000000000001 maps to 0x3F3E3D3C3B3A3907, you have to make sure that no other value can map to 0x3F3E3D3C3B3A3907.
A very easy way to obfuscate a number is to convert it to a byte array and then map each byte individually by XORing it with some constant value. For example, if you pick the value 0xC0 as your XOR value, then the value 1 would map to 0xC0C0C0C0C0C0C0C1. Of course, 2 maps to 0xC0C0C0C0C0C0C0C2, so the sequential problem doesn't really go away.
A better way is to create a table that converts byte values to some "random" value. That is, the value 0x01 maps to 0xF8, 0x02 maps to 0x07, etc. There's no sequence to the mapping. Rather, we generate a random mapping table so that there is a one-to-one mapping from one value to another. And, to de-obfuscate things, we create the reverse mapping.
Creating the encoding table
Creating the encoding table is a simple matter of generating an array of 256 bytes, in order, and then scrambling them. Here's code to do it:
static byte[] CreateEncodingTable()
{
// Create an array of 0..255
byte[] b = new byte[256];
for (int i = 0; i < b.Length; ++i)
{
b[i] = (byte)i;
}
// Now randomize the array
Random rnd = new Random();
for (int i = 0; i < b.Length; ++i)
{
int swapWith = rnd.Next(b.Length);
byte temp = b[i];
b[i] = b[swapWith];
b[swapWith] = temp;
}
return b;
}
You might be tempted to put that code in your program to have it create the encoding table at program startup. That would be a mistake because the random number generator will be initialized with a different value every time. You could change the code so that it provides a seed for the random number generator (i.e. Random rnd = new Random(1234);), but even that's a bad idea. Why? Because the implementation of the random number generator might change in the next version of .NET and then values that were generated with the old version won't be decoded properly.
The best way to handle this is to write a little program that creates the encoding table and then outputs a C# code snippet that you can include in your program. You just need a SaveEncodingTable method:
static void SaveEncodingTable(byte[] b, string filename)
{
using (var outfile = new StreamWriter(filename))
{
outfile.WriteLine("private static readonly byte[] EncodingTable = new byte[]");
outfile.WriteLine("{");
for (int i = 0; i < b.Length; ++i)
{
outfile.Write(" 0x{0:X2},", b[i]);
if (((i + 1) % 8) == 0)
{
outfile.WriteLine();
}
}
outfile.WriteLine("};");
}
}
Then, you create the encoding table and save it like this:
var b = CreateEncodingTable(); SaveEncodingTable(b, "snippet.cs");
You can then include the encoding table in your class that encodes and decodes keys.
The KeyCoder class
We're going to end up with a static KeyCoder class that handles encoding and decoding of keys. It will have two primary API functions: KeyToString and StringToKey. Before we can start building those, though, we need to add the generated encoding table and write code that will create the decoding table. Here's the beginnings of the class, which includes the encoding table:
public static class KeyCoder
{
// Program-generated encoding table.
// DO NOT CHANGE!
private static readonly byte[] EncodingTable = new byte[]
{
0x3C, 0x88, 0x1B, 0x79, 0x51, 0x13, 0xE6, 0x4A,
0x22, 0x39, 0x73, 0x2D, 0xC1, 0x7E, 0x83, 0x64,
0xAD, 0x01, 0x5C, 0xCF, 0xC8, 0xEF, 0x7B, 0x0F,
0x16, 0x43, 0x4C, 0xFB, 0x0D, 0xC3, 0xAA, 0x7D,
0x1C, 0x2E, 0x52, 0x85, 0x90, 0x81, 0x11, 0x05,
0xA4, 0x56, 0xE7, 0x53, 0x25, 0x65, 0x48, 0xE0,
0x6D, 0x76, 0x29, 0x21, 0xF3, 0xFC, 0x78, 0x1E,
0xBA, 0xA0, 0x0A, 0xA3, 0x5A, 0xAB, 0x36, 0xCE,
0x8E, 0xAC, 0x44, 0x8D, 0x2B, 0xFD, 0x95, 0xA6,
0x46, 0x42, 0xD2, 0x80, 0xEA, 0xDD, 0xD0, 0x24,
0x1D, 0xDA, 0xA7, 0x75, 0x9D, 0x5F, 0x31, 0x55,
0xC4, 0x5E, 0x9A, 0x66, 0x74, 0xD1, 0xB0, 0xE9,
0x8B, 0xD7, 0x35, 0x82, 0x26, 0x72, 0x96, 0x6F,
0x77, 0xCD, 0x9B, 0xD3, 0xBD, 0x63, 0x41, 0x45,
0x84, 0x9E, 0x0B, 0xEE, 0x28, 0xF4, 0x03, 0xCC,
0x0C, 0xF0, 0xFF, 0xDB, 0x67, 0x89, 0x10, 0xEB,
0xA1, 0xC6, 0x86, 0x08, 0xC2, 0xE1, 0xC9, 0x27,
0x60, 0xFA, 0x92, 0xF9, 0x2F, 0xAE, 0x91, 0xA9,
0xE2, 0xBC, 0xB3, 0x57, 0x2C, 0x34, 0xF8, 0xC7,
0x33, 0xD5, 0xE4, 0xB4, 0x69, 0x97, 0xDC, 0xCB,
0xB1, 0x8A, 0xF1, 0x98, 0x3F, 0x94, 0xD6, 0x40,
0x32, 0xB8, 0xEC, 0x02, 0x8C, 0xB6, 0x50, 0xF2,
0xD9, 0x9C, 0x4F, 0x62, 0xBE, 0x09, 0xC0, 0xDF,
0x5D, 0x3E, 0x23, 0xB5, 0x37, 0xB9, 0x59, 0xF6,
0xDE, 0x5B, 0xD4, 0x00, 0x61, 0x19, 0x54, 0x14,
0x2A, 0xC5, 0x4D, 0x68, 0x71, 0xED, 0x06, 0x20,
0x93, 0xF5, 0x0E, 0xB2, 0xFE, 0x15, 0xBF, 0x7A,
0x8F, 0x17, 0x6C, 0x30, 0xAF, 0x87, 0x3A, 0xE8,
0xA8, 0xB7, 0xA2, 0x1F, 0x6E, 0x18, 0x38, 0x7F,
0x04, 0x9F, 0x6B, 0xA5, 0x1A, 0x47, 0x3B, 0x7C,
0x3D, 0x4E, 0x99, 0x70, 0x07, 0x12, 0xBB, 0x58,
0x6A, 0x4B, 0xD8, 0x49, 0xF7, 0xE5, 0xCA, 0xE3,
};
private static readonly byte[] DecodingTable;
static KeyCoder()
{
// initialize the decoding table
throw new NotImplementedException();
}
// Return a base64-encoded key string
public static string KeyToString(ulong val)
{
throw new NotImplementedException();
}
// Decode a base64-encoded key string
public static ulong StringToKey(ulong val)
{
throw new NotImplementedException();
}
// Methods to encode and decode keys
public static ulong EncodeKey(ulong val)
{
throw new NotImplementedException();
}
public static ulong DecodeKey(ulong val)
{
throw new NotImplementedException();
}
}
Most of the class isn't implemented, but you can see where it's going. The generated encoding table is there, and we've already mostly written the KeyToString and StringToKey methods. They will be subtlety different from what we created earlier, so I haven't included them in the class yet.
We just have to write code that initializes the decoding table, and then handle encoding and decoding the keys. That is, turning 0x0000000000000001 into a "random" looking value like 0x3F3E3D3C3B3A3907.
I made the EncodeKey and DecodeKey methods public so that we can test them independently of the KeyToString and KeyToString methods.
Initializing the decoding table is very easy. Just go through the EncodingTable, and for each value read, put the index into the corresponding value of DecodingTable. For example, EncodingTable[0] contains the value 0x3C, so we'd want to put the value 0x00 at DecodingTable[0x3C]. Here's the new static constructor with the code.
static KeyCoder()
{
// initialize the decoding table
DecodingTable = new byte[256];
for (int i = 0; i < 256; ++i)
{
byte b = EncodingTable[i];
DecodingTable[b] = (byte)i;
}
}
Encoding and decoding
The trivially simple encoding method is to look up each byte of the 64-bit key in the table, and create a new value with the translated values. That is, if your 64-bit key is 0x0000000000000001, then the new encoded value would be 0x3C3C3C3C3C3C3C88. Decoding is the reverse: looking up each byte from the encoded value in the decoding table. Using that method, EncodeKey and DecodeKey become:
public static ulong EncodeKey(ulong val)
{
// Get the 64-bit value as bytes
byte[] bytes = BitConverter.GetBytes(val);
// Replace each byte with the corresponding value from the encoding table.
for (int i = 0; i < bytes.Length; ++i)
{
byte b = EncodingTable[bytes[i]];
bytes[i] = b;
}
return BitConverter.ToUInt64(bytes, 0);
}
public static ulong DecodeKey(ulong val)
{
// Get the 64-bit value as bytes
byte[] bytes = BitConverter.GetBytes(val);
// Replace each byte with the corresponding value from the decoding table.
for (int i = 0; i < bytes.Length; ++i)
{
byte b = DecodingTable[bytes[i]];
bytes[i] = b;
}
return BitConverter.ToUInt64(bytes, 0);
}
Let's make sure that those methods are symmetrical. That is, if we write:
ulong key = 1; ulong encoded = EncodeKey(key); ulong decoded = DecodeKey(encoded);
Then at this point, decoded should be equal to key.
I can modify the code I wrote to test the base64 encoding so that it calls EncodeKey and DecodeKey:
for (; ; )
{
Console.WriteLine("Enter value to be encoded. Blank line to exit.");
Console.Write(">");
string sval = Console.ReadLine();
if (string.IsNullOrWhiteSpace(sval))
break;
ulong val;
if (ulong.TryParse(sval, out val))
{
ulong encoded = KeyCoder.EncodeKey(val);
ulong decoded = KeyCoder.DecodeKey(encoded);
Console.Write("{0:X8} -> {1:X8} -> {2:X8}", val, encoded, decoded);
if (decoded != val)
{
Console.Write(" ERROR!");
}
Console.WriteLine();
Console.WriteLine();
}
}
Sample output shows that keys make the round trip just fine:
Enter value to be encoded. Blank line to exit. >1 00000001 -> 3C3C3C3C3C3C3C88 -> 00000001 Enter value to be encoded. Blank line to exit. >33 00000021 -> 3C3C3C3C3C3C3C2E -> 00000021 >123456789012345678 1B69B4BA630F34E -> 88C0B480D66D70D0 -> 1B69B4BA630F34E
Putting it all together
Now all we have to do is modify the KeyToString and StringToKey methods so that they'll call the EncodeKey and DecodeKey methods, as appropriate:
public static string KeyToString(ulong val)
{
// encode the key
ulong encodedKey = EncodeKey(val);
// get bytes
byte[] keyBytes = BitConverter.GetBytes(encodedKey);
// get base64 value
string keyString = Convert.ToBase64String(keyBytes);
// The base64 encoding has a trailing = sign, and + and - characters.
// Strip the trailing =.
keyString = keyString.Substring(0, keyString.Length - 1);
// convert + to - (dash) and / to _ (underscore)
keyString = keyString.Replace('+', '-');
keyString = keyString.Replace('/', '_');
return keyString;
}
public static ulong StringToKey(string keyString)
{
// convert - to +, and _ to /
keyString = keyString.Replace('-', '+');
keyString = keyString.Replace('_', '/');
// add the trailing =
keyString += '=';
// convert to bytes
byte[] keyBytes = Convert.FromBase64String(keyString);
// get the encoded key
ulong encodedKey = BitConverter.ToUInt64(keyBytes, 0);
// and decode it
return DecodeKey(encodedKey);
}
The only real difference between those two methods and the original versions is that KeyToString calls EncodeKey before doing the base64 conversion, and StringToKey calls DecodeKey to get the decoded value before returning.
If we again modify our test program so that it calls KeyToString and StringToKey, we can verify that the code works as expected. In the interest of brevity, I've only included the part of the code that actually does the test.
if (ulong.TryParse(sval, out val))
{
string keyString = KeyCoder.KeyToString(val);
ulong key = KeyCoder.StringToKey(keyString);
Console.Write("{0:X16} -> {1} -> {2:X16}", val, keyString, key);
if (key != val)
{
Console.Write(" ERROR!");
}
Console.WriteLine();
Console.WriteLine();
}
Output from the program verifies that the keys make the round trip okay.
Enter value to be encoded. Blank line to exit. >0 0000000000000000 -> PDw8PDw8PDw -> 0000000000000000 Enter value to be encoded. Blank line to exit. >1 0000000000000001 -> iDw8PDw8PDw -> 0000000000000001 Enter value to be encoded. Blank line to exit. >2 0000000000000002 -> Gzw8PDw8PDw -> 0000000000000002 Enter value to be encoded. Blank line to exit. >3 0000000000000003 -> eTw8PDw8PDw -> 0000000000000003 Enter value to be encoded. Blank line to exit. >99042187365 000000170F5FD865 -> co_pZA88PDw -> 000000170F5FD865 Enter value to be encoded. Blank line to exit. >
The keys do make the round trip, but there's a problem. The keys for 0, 1, 2, and 3 are, respectively:
PDw8PDw8PDw iDw8PDw8PDw Gzw8PDw8PDw eTw8PDw8PDw
All but the first two characters are identical. To somebody examining the keys for a pattern, it would be immediately obvious what's happening. Even the value 99042187365 exhibits the pattern. Its key, co_pZA88PDw, shares the last four characters with the others. It's obvious from studying the keys that they reflect an increasing value.
One more obfuscation
Remember back when I used XOR to create the simple encoder? We can implement a variant of that uses XOR to remove the regularity that you see in the encoded keys.
We can't just XOR with a constant value, as that would simply change the pattern of the keys. Instead, we'll use the first byte of the key as the XOR value, and increase it by one for each of the following bytes. The result is that in the encoded key, the low byte is the initial XOR key.
It just takes a small modification to the EncodeKey and DecodeKey methods to initialize the XOR value and then apply it as we go through the bytes in the number. The code below, which includes the entire KeyCoder class, has the modified methods.
public static class KeyCoder
{
// Program-generated encoding table.
// DO NOT CHANGE!
private static readonly byte[] EncodingTable = new byte[]
{
0x3C, 0x88, 0x1B, 0x79, 0x51, 0x13, 0xE6, 0x4A,
0x22, 0x39, 0x73, 0x2D, 0xC1, 0x7E, 0x83, 0x64,
0xAD, 0x01, 0x5C, 0xCF, 0xC8, 0xEF, 0x7B, 0x0F,
0x16, 0x43, 0x4C, 0xFB, 0x0D, 0xC3, 0xAA, 0x7D,
0x1C, 0x2E, 0x52, 0x85, 0x90, 0x81, 0x11, 0x05,
0xA4, 0x56, 0xE7, 0x53, 0x25, 0x65, 0x48, 0xE0,
0x6D, 0x76, 0x29, 0x21, 0xF3, 0xFC, 0x78, 0x1E,
0xBA, 0xA0, 0x0A, 0xA3, 0x5A, 0xAB, 0x36, 0xCE,
0x8E, 0xAC, 0x44, 0x8D, 0x2B, 0xFD, 0x95, 0xA6,
0x46, 0x42, 0xD2, 0x80, 0xEA, 0xDD, 0xD0, 0x24,
0x1D, 0xDA, 0xA7, 0x75, 0x9D, 0x5F, 0x31, 0x55,
0xC4, 0x5E, 0x9A, 0x66, 0x74, 0xD1, 0xB0, 0xE9,
0x8B, 0xD7, 0x35, 0x82, 0x26, 0x72, 0x96, 0x6F,
0x77, 0xCD, 0x9B, 0xD3, 0xBD, 0x63, 0x41, 0x45,
0x84, 0x9E, 0x0B, 0xEE, 0x28, 0xF4, 0x03, 0xCC,
0x0C, 0xF0, 0xFF, 0xDB, 0x67, 0x89, 0x10, 0xEB,
0xA1, 0xC6, 0x86, 0x08, 0xC2, 0xE1, 0xC9, 0x27,
0x60, 0xFA, 0x92, 0xF9, 0x2F, 0xAE, 0x91, 0xA9,
0xE2, 0xBC, 0xB3, 0x57, 0x2C, 0x34, 0xF8, 0xC7,
0x33, 0xD5, 0xE4, 0xB4, 0x69, 0x97, 0xDC, 0xCB,
0xB1, 0x8A, 0xF1, 0x98, 0x3F, 0x94, 0xD6, 0x40,
0x32, 0xB8, 0xEC, 0x02, 0x8C, 0xB6, 0x50, 0xF2,
0xD9, 0x9C, 0x4F, 0x62, 0xBE, 0x09, 0xC0, 0xDF,
0x5D, 0x3E, 0x23, 0xB5, 0x37, 0xB9, 0x59, 0xF6,
0xDE, 0x5B, 0xD4, 0x00, 0x61, 0x19, 0x54, 0x14,
0x2A, 0xC5, 0x4D, 0x68, 0x71, 0xED, 0x06, 0x20,
0x93, 0xF5, 0x0E, 0xB2, 0xFE, 0x15, 0xBF, 0x7A,
0x8F, 0x17, 0x6C, 0x30, 0xAF, 0x87, 0x3A, 0xE8,
0xA8, 0xB7, 0xA2, 0x1F, 0x6E, 0x18, 0x38, 0x7F,
0x04, 0x9F, 0x6B, 0xA5, 0x1A, 0x47, 0x3B, 0x7C,
0x3D, 0x4E, 0x99, 0x70, 0x07, 0x12, 0xBB, 0x58,
0x6A, 0x4B, 0xD8, 0x49, 0xF7, 0xE5, 0xCA, 0xE3,
};
private static readonly byte[] DecodingTable;
static KeyCoder()
{
DecodingTable = new byte[256];
for (int i = 0; i < 256; ++i)
{
byte b = EncodingTable[i];
DecodingTable[b] = (byte)i;
}
}
public static string KeyToString(ulong val)
{
// encode the key
ulong encodedKey = EncodeKey(val);
// get bytes
byte[] keyBytes = BitConverter.GetBytes(encodedKey);
// get base64 value
string keyString = Convert.ToBase64String(keyBytes);
// The base64 encoding has a trailing = sign, and + and - characters.
// Strip the trailing =.
keyString = keyString.Substring(0, keyString.Length - 1);
// convert + to - (dash) and / to _ (underscore)
keyString = keyString.Replace('+', '-');
keyString = keyString.Replace('/', '_');
return keyString;
}
public static ulong StringToKey(string keyString)
{
// convert - to +, and _ to /
keyString = keyString.Replace('-', '+');
keyString = keyString.Replace('_', '/');
// add the trailing =
keyString += '=';
// convert to bytes
byte[] keyBytes = Convert.FromBase64String(keyString);
// get the encoded key
ulong encodedKey = BitConverter.ToUInt64(keyBytes, 0);
// and decode it
return DecodeKey(encodedKey);
}
public static ulong EncodeKey(ulong val)
{
// Get the 64-bit value as bytes
byte[] bytes = BitConverter.GetBytes(val);
// Replace each byte with the corresponding value from the encoding table, after XOR.
byte xorValue = 0;
for (int i = 0; i < bytes.Length; ++i)
{
byte b = EncodingTable[bytes[i]];
if (i == 0)
{
xorValue = b;
}
else
{
b ^= xorValue;
}
bytes[i] = b;
++xorValue;
}
return BitConverter.ToUInt64(bytes, 0);
}
public static ulong DecodeKey(ulong val)
{
// Get the 64-bit value as bytes
byte[] bytes = BitConverter.GetBytes(val);
// Replace each byte with the corresponding value from the decoding table, after XOR.
byte xorValue = 0;
for (int i = 0; i < bytes.Length; ++i)
{
byte b = bytes[i];
if (i == 0)
{
xorValue = b;
}
else
{
b ^= xorValue;
}
b = DecodingTable[b];
bytes[i] = b;
++xorValue;
}
return BitConverter.ToUInt64(bytes, 0);
}
}
And if you test the program you'll see that the regularity in sequential keys is gone.
Enter value to be encoded. Blank line to exit. >0 0000000000000000 -> PAECA3x9fn8 -> 0000000000000000 Enter value to be encoded. Blank line to exit. >1 0000000000000001 -> iLW2t7CxsrM -> 0000000000000001 Enter value to be encoded. Blank line to exit. >2 0000000000000002 -> GyAhIiMcHR4 -> 0000000000000002 Enter value to be encoded. Blank line to exit. >3 0000000000000003 -> eUZHQEFCQ7w -> 0000000000000003 Enter value to be encoded. Blank line to exit. >1024 0000000000000400 -> PGwCA3x9fn8 -> 0000000000000400 Enter value to be encoded. Blank line to exit. >65536 0000000000010000 -> PAG2A3x9fn8 -> 0000000000010000
There is still some regularity, though. The last two values are powers of two, with hex values 0x0400 and 0x10000 respectively. They share common patterns in the low bits, and as a result will have common trailing characters in their keys. In general with this approach, the repetition period is 256. That is, the base64 representation of the key 0 is "PAECA3x9fn8", and the string for 256 is "PLUCA3x9fn8". Whether that's good enough for your purposes is up to you.
There are a few other things you can do to further obfuscate the keys, but you have to be careful not to do something that will violate the uniqueness property.
One possibility is to use the previous byte's value as an additional offset into
One possibility is to use an offset when indexing into the encoding table. Something like this, using the first version of EncodeKey:
public static ulong EncodeKey(ulong val)
{
// Get the 64-bit value as bytes
byte[] bytes = BitConverter.GetBytes(val);
byte tableOffset = 0;
// Replace each byte with the corresponding value from the encoding table.
for (int i = 0; i < bytes.Length; ++i)
{
byte b = EncodingTable[(byte)(bytes[i] + tableOffset)];
bytes[i] = b;
tableOffset = b;
}
return BitConverter.ToUInt64(bytes, 0);
}
Using something like that in combination with the XOR should remove or at least reduce the regularity you see in the keys, but I haven't tested it. Decoding, of course, would need a similar modification.
Obfuscation is not encryption
I've been careful in my discussion to use the words "encoding" or "obfuscation" rather than "encryption." The technique described in this article merely makes it more difficult to determine that the underlying key is assigned sequentially, or to determine the real key value from the resulting base64 encoded string. Some might say that this is a simple form for encryption. Perhaps it is. But it is not at all secure. This obfuscation scheme could be "broken" fairly easily by a determined attacker.
That said, this scheme creates random-looking fixed-length keys that are similar in form to the keys that people are used to seeing from YouTube and other such sites. People understand that they are unique values that reference particular resources. The obfuscation scheme not only creates keys that look like those that people are already used to seeing, but also discourage all but the most determined from trying to decode them to find the underlying sequence.



