I’ve wanted to “factory reset” my old Android phones for years but hadn’t found a good way of exporting text messages including the embedded images and videos. Some solutions involve rooting your phone, and I didn’t want to do that. Nor did I want to use an app that copied to Gmail or the cloud. And I wanted to know exactly what the app’s code was doing. So I found the MessageBackup repo on GitHub and used it as a starting point.
In my iteration I decided to fetch SMS and MMS messages separately and explicitly instead of fetching “conversations”. With the latter you have to deal with different types of objects (SMS and MMS) in the same stream, which I found confusing. We encode all binary data (like images or videos) to base64. And we preserve as much of the original data structure as possible by simply converting the fields and values to JSON objects using the following function. It loops over the columns/fields at the current cursor location, and adds them into an existing JSON object.
private void addJsonFields(JSONObject target, Cursor source) {
for(int i = 0; i < source.getColumnCount(); i++) {
try {
switch (source.getType(i)) {
case Cursor.FIELD_TYPE_BLOB:
target.put(source.getColumnName(i), source.getBlob(i));
break;
case Cursor.FIELD_TYPE_FLOAT:
target.put(source.getColumnName(i), source.getFloat(i));
break;
case Cursor.FIELD_TYPE_INTEGER:
// Cursor reports an int type, but 4 bytes might not be enough,
// so instead of getInt() use getLong() to be safe
target.put(source.getColumnName(i), source.getLong(i));
break;
case Cursor.FIELD_TYPE_NULL:
target.put(source.getColumnName(i), null);
break;
case Cursor.FIELD_TYPE_STRING:
target.put(source.getColumnName(i), source.getString(i));
break;
}
} catch (Exception e) {
Log.e(TAG, Log.getStackTraceString(e));
}
}
}
The project is android-text-message-backup on GitHub, in case you want to take a quick look before proceeding.
In order to use the app, follow these steps. I assume you’re familiar with Android development and Android Studio.
- Install Android Studio
- Android Studio seems to change project structure every year, so create a new project
- Select Empty Activity
- On the next screen select “API 16: Android 4.1 (Jelly Bean)” as the Minimum SDK. Unless of course you need something earlier and are up for some light development work.
- Fill out the rest of the fields
- From the src folder
- Copy my code into your
MainApplication.javafile - Modify your project’s
AndrioidManifest.xmlto closely match mine. Note permissions and default activity launch stuff - Modify your
strings.xmlresource file and copy in the 4 resources from mine - Copy my
activity_main.xmllayout file contents into yours
- Copy my code into your
- Hook your phone up to your computer using a USB cable
- Ensure Android Studio sees it
- Build and Run the app on your phone. Be sure to grant the app SMS and Storage privs.
- Press the Backup button on the app
- Watch the console log in Android Studio to see it making progress
- Note the location of the JSON that shows on your phone’s screen
- Use the Device Explorer to navigate to the correct folder
- Right click the file and Save-As to copy to your computer
I have an accompanying python script that decodes the base64 into image and video files and makes a single html file for browsing the messages and images. See src/gen-html.py. But you can always write your own.
I relied heavily on Stack Overflow for guidance, specifically this question: https://stackoverflow.com/questions/3012287/how-to-read-mms-data-in-android.
Since I pieced together many different things to get this working, it’s worth summarizing them case you’re looking for a working example of how to do something in Android/Java:
- Fetch and iterate over all SMS and MMS messages
- Fetch image, video and other binary data from MMS messages
- Encode binary data from an MMS part to base64
- Fetch participants of a group text/MMS message
- Properly encode data to JSON, whether as strings, objects, arrays
- Write file to the SDCard or internal storage
Again, you can find the code in my android-text-message-backup project on Github. So far I’ve tested it against the following phones. The export for the Motorola was about 300MB!
- Samsung Galaxy S Relay 4G - Android 4.4.4
- Motorola Moto G, 3rd gen - Android 6.0.1
- Nokia 6.1 - Android 9
Hope it helps! And if you want to browse the Java code now, here ya go:
package com.alanszlosek.messagebackup;
import android.content.ContentResolver;
import android.content.res.Resources;
import android.os.Bundle;
import android.view.View;
import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.text.MessageFormat;
import org.json.JSONArray;
import org.json.JSONObject;
import android.net.Uri;
import android.os.Environment;
import android.database.Cursor;
import android.util.Base64;
import android.util.Log;
import android.widget.ProgressBar;
import android.widget.TextView;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.content.ContextCompat;
public class MainActivity extends AppCompatActivity {
private static final String TAG = "MessageBackup";
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
}
private void addJsonFields(JSONObject target, Cursor source) {
for(int i = 0; i < source.getColumnCount(); i++) {
try {
switch (source.getType(i)) {
case Cursor.FIELD_TYPE_BLOB:
target.put(source.getColumnName(i), source.getBlob(i));
break;
case Cursor.FIELD_TYPE_FLOAT:
target.put(source.getColumnName(i), source.getFloat(i));
break;
case Cursor.FIELD_TYPE_INTEGER:
// Cursor reports an int type, but 4 bytes might not be enough,
// so instead of getInt() use getLong() to be safe
target.put(source.getColumnName(i), source.getLong(i));
break;
case Cursor.FIELD_TYPE_NULL:
target.put(source.getColumnName(i), null);
break;
case Cursor.FIELD_TYPE_STRING:
target.put(source.getColumnName(i), source.getString(i));
break;
}
} catch (Exception e) {
Log.e(TAG, Log.getStackTraceString(e));
}
}
}
private byte[] readBytes(InputStream inputStream) throws IOException {
// this dynamically extends to take the bytes you read
ByteArrayOutputStream byteBuffer = new ByteArrayOutputStream();
// this is storage overwritten on each iteration with bytes
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
// we need to know how may bytes were read to write them to the byteBuffer
int len = 0;
while ((len = inputStream.read(buffer)) != -1) {
byteBuffer.write(buffer, 0, len);
}
// and then we can return your byte array.
return byteBuffer.toByteArray();
}
// COPIED
public void startBackup(View view) throws Exception {
Resources res = getResources();
TextView tv = (TextView) this.findViewById(R.id.textView);
ProgressBar mainProgressBar = (ProgressBar) this.findViewById(R.id.progressBar1);
File[] externalStorageVolumes = ContextCompat.getExternalFilesDirs(getApplicationContext(), null);
File newxmlfile;
String filename = "messages-" + (System.currentTimeMillis() / 1000) + ".json";
mainProgressBar.setProgress(0);
if (Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) {
Log.e(TAG, "External storage is mounted and writeable");
newxmlfile = new File(externalStorageVolumes[0], filename);
} else {
newxmlfile = new File(this.getFilesDir(), filename);
}
tv.append(String.format(res.getString(R.string.debug), newxmlfile.getAbsolutePath()));
FileOutputStream fileos = null;
try {
newxmlfile.createNewFile();
fileos = new FileOutputStream(newxmlfile);
} catch (Exception e) {
Log.e(TAG, "can't create FileOutputStream");
return;
}
/*
conversations
sms-mms
*/
/*
{
'sms-mms': [
{
'_id': 'bla',
'thread_id': 'bla',
'messages': [
{
'_id': 'bla',
}
]
}
],
'drafts': [
]
*/
// Code pieced together from this great SO answer: https://stackoverflow.com/questions/3012287/how-to-read-mms-data-in-android
JSONObject jsonObject1;
JSONArray jsonArray1;
JSONArray jsonArray2;
ContentResolver contentResolver = getContentResolver();
fileos.write("{".getBytes());
// TODO: handle drafts too
boolean hasItems1;
Cursor cursor1;
Cursor cursor2;
cursor1 = contentResolver.query(Uri.parse("content://sms/"), null, null, null, null);
cursor2 = contentResolver.query(Uri.parse("content://mms/"), null, null, null, null);
cursor1.moveToFirst();
cursor2.moveToFirst();
int numMessages = cursor1.getCount() + cursor2.getCount();
mainProgressBar.setMax(numMessages);
Log.e(TAG, "num messages " + numMessages);
int progress = 0;
// cursor1 = contentResolver.query(Uri.parse("content://sms/"), null, null, null, null);
fileos.write("\"sms\":[".getBytes());
if (!cursor1.moveToFirst()) {
Log.e(TAG, "No sms to export");
} else {
hasItems1 = false;
do {
if (hasItems1) {
fileos.write(",".getBytes());
}
jsonObject1 = new JSONObject();
addJsonFields(jsonObject1, cursor1);
fileos.write(jsonObject1.toString().getBytes());
hasItems1 = true;
progress++;
mainProgressBar.setProgress(progress);
} while (cursor1.moveToNext());
cursor1.close();
}
fileos.write("],".getBytes());
//cursor2 = contentResolver.query(Uri.parse("content://mms/"), null, null, null, null);
fileos.write("\"mms\":[".getBytes());
if (!cursor2.moveToFirst()) {
Log.e(TAG, "No mms to export");
} else {
hasItems1 = false;
do { // looping over mms
String id = cursor2.getString(cursor2.getColumnIndex(("_id")));
byte[] b = new byte[3000]; // buffer for reading MMS part data (images, video)
jsonObject1 = new JSONObject();
jsonArray1 = new JSONArray();
jsonArray2 = new JSONArray();
addJsonFields(jsonObject1, cursor2);
jsonObject1.put("parts", jsonArray1);
jsonObject1.put("addresses", jsonArray2);
// Get sender, just push each address onto an addresses list for now
String selectionAddress = new String("msg_id=" + id);
Uri uriAddress = Uri.parse(MessageFormat.format("content://mms/{0}/addr", id));
Cursor addresses = getContentResolver().query(uriAddress, null, selectionAddress, null, null);
if (addresses.moveToFirst()) {
do {
JSONObject jsonObject3 = new JSONObject();
addJsonFields(jsonObject3, addresses);
jsonArray2.put(jsonObject3);
} while (addresses.moveToNext());
}
if (addresses != null) {
addresses.close();
}
// Now get MMS parts
String selectionPart = "mid=" + id;
Cursor parts = getContentResolver().query(Uri.parse("content://mms/part"), null, selectionPart, null, null);
if (!parts.moveToFirst()) {
Log.e(TAG, "No parts for mid " + id);
parts.close();
continue;
}
if (hasItems1) {
fileos.write(",".getBytes());
}
do { // looping over parts in this mms message
JSONObject jsonObject2 = new JSONObject();
// TODO: feel we need to append text parts in order
String partId = parts.getString(parts.getColumnIndex("_id"));
String partType = parts.getString(parts.getColumnIndex("ct"));
addJsonFields(jsonObject2, parts);
jsonArray1.put(jsonObject2);
if ("text/plain".equals(partType)) {
String data = parts.getString(parts.getColumnIndex("_data"));
String body;
if (data != null) {
InputStream is = null;
StringBuilder sb = new StringBuilder();
try {
is = getContentResolver().openInputStream(Uri.parse("content://mms/part/" + partId));
if (is != null) {
InputStreamReader isr = new InputStreamReader(is, "UTF-8");
BufferedReader reader = new BufferedReader(isr);
String temp = reader.readLine();
while (temp != null) {
sb.append(temp);
temp = reader.readLine();
}
}
} catch (Exception e) {
Log.e(TAG, "Exception while fetching contents for message " + id);
if (is != null) {
is.close();
}
continue;
}
is.close();
body = sb.toString();
} else {
body = parts.getString(parts.getColumnIndex("text"));
}
jsonObject2.put("body", body);
} else if ("image/jpeg".equals(partType) || "image/bmp".equals(partType) ||
"image/gif".equals(partType) || "image/jpg".equals(partType) ||
"image/png".equals(partType) ||
"video/3gpp".equals(partType) ||
"video/mp4".equals(partType) ) {
InputStream is = null;
Log.e(TAG, "Fetching data for part " + partId);
try {
is = contentResolver.openInputStream(Uri.parse("content://mms/part/" + partId));
} catch (FileNotFoundException e) {
Log.e(TAG, "Part " + partId + " data not found for message " + id);
if (is != null) {
is.close();
}
continue;
} catch (Exception e) {
Log.e(TAG, "Exception while handling data for part " + partId);
Log.e(TAG, Log.getStackTraceString(e));
if (is != null) {
is.close();
}
continue;
}
// TODO: perhaps try incremental base64 encoding
jsonObject2.put("base64", Base64.encodeToString(readBytes(is), Base64.DEFAULT));
if (is != null) {
is.close();
}
}
} while (parts.moveToNext());
parts.close();
fileos.write(jsonObject1.toString().getBytes());
hasItems1 = true;
progress++;
mainProgressBar.setProgress(progress);
} while (cursor2.moveToNext());
cursor2.close();
fileos.write("]".getBytes());
}
fileos.write("}".getBytes());
fileos.close();
Log.e(TAG, "Done");
}
}