Patches redux-persist and redux-deep-persist with debounced persists
Our app changes redux state very, very often. As our undo/redo history grows, the calls to persist state start to take in the 100ms range, due to a the deep cloning of the history. This causes very noticeable performance lag.
The deep cloning is required because we need to blacklist certain items in redux from being persisted (e.g. the app's connection status).
Debouncing the whole process of persistence is a simple and effective solution. Unfortunately, `redux-persist` dropped `debounce` between v4 and v5, replacing it with `throttle`. `throttle`, instead of delaying the expensive action until a period of X ms of inactivity, simply ensures the action is executed at least every X ms. Of course, this does not fix our performance issue.
The patch is very simple. It adds a `debounce` argument - a number of milliseconds - and debounces `redux-persist`'s `update()` method (provided by `createPersistoid`) by that many ms.
Before this, I also tried writing a custom storage adapter for `redux-persist` to debounce the calls to `localStorage.setItem()`. While this worked and was far less invasive, it doesn't actually address the issue. It turns out `setItem()` is a very fast part of the process.
We use `redux-deep-persist` to simplify the `redux-persist` configuration, which can get complicated when you need to blacklist or whitelist deeply nested state. There is also a patch here for that library because it uses the same types as `redux-persist`.
Unfortunately, the last release of `redux-persist` used a package `flat-stream` which was malicious and has been removed from npm. The latest commits to `redux-persist` (about 1 year ago) do not build; we cannot use the master branch. And between the last release and last commit, the changes have all been breaking.
Patching this last release (about 3 years old at this point) directly is far simpler than attempting to fix the upstream library's master branch or figuring out an alternative to the malicious and now non-existent dependency.
2022-11-12 09:41:48 +00:00
|
|
|
diff --git a/node_modules/redux-persist/es/createPersistoid.js b/node_modules/redux-persist/es/createPersistoid.js
|
2022-11-12 10:47:05 +00:00
|
|
|
index 8b43b9a..ffe8d3f 100644
|
Patches redux-persist and redux-deep-persist with debounced persists
Our app changes redux state very, very often. As our undo/redo history grows, the calls to persist state start to take in the 100ms range, due to a the deep cloning of the history. This causes very noticeable performance lag.
The deep cloning is required because we need to blacklist certain items in redux from being persisted (e.g. the app's connection status).
Debouncing the whole process of persistence is a simple and effective solution. Unfortunately, `redux-persist` dropped `debounce` between v4 and v5, replacing it with `throttle`. `throttle`, instead of delaying the expensive action until a period of X ms of inactivity, simply ensures the action is executed at least every X ms. Of course, this does not fix our performance issue.
The patch is very simple. It adds a `debounce` argument - a number of milliseconds - and debounces `redux-persist`'s `update()` method (provided by `createPersistoid`) by that many ms.
Before this, I also tried writing a custom storage adapter for `redux-persist` to debounce the calls to `localStorage.setItem()`. While this worked and was far less invasive, it doesn't actually address the issue. It turns out `setItem()` is a very fast part of the process.
We use `redux-deep-persist` to simplify the `redux-persist` configuration, which can get complicated when you need to blacklist or whitelist deeply nested state. There is also a patch here for that library because it uses the same types as `redux-persist`.
Unfortunately, the last release of `redux-persist` used a package `flat-stream` which was malicious and has been removed from npm. The latest commits to `redux-persist` (about 1 year ago) do not build; we cannot use the master branch. And between the last release and last commit, the changes have all been breaking.
Patching this last release (about 3 years old at this point) directly is far simpler than attempting to fix the upstream library's master branch or figuring out an alternative to the malicious and now non-existent dependency.
2022-11-12 09:41:48 +00:00
|
|
|
--- a/node_modules/redux-persist/es/createPersistoid.js
|
|
|
|
|
+++ b/node_modules/redux-persist/es/createPersistoid.js
|
2022-11-12 10:47:05 +00:00
|
|
|
@@ -1,11 +1,13 @@
|
|
|
|
|
import { KEY_PREFIX, REHYDRATE } from './constants';
|
|
|
|
|
// @TODO remove once flow < 0.63 support is no longer required.
|
|
|
|
|
export default function createPersistoid(config) {
|
|
|
|
|
+ console.log(config)
|
|
|
|
|
// defaults
|
|
|
|
|
var blacklist = config.blacklist || null;
|
Patches redux-persist and redux-deep-persist with debounced persists
Our app changes redux state very, very often. As our undo/redo history grows, the calls to persist state start to take in the 100ms range, due to a the deep cloning of the history. This causes very noticeable performance lag.
The deep cloning is required because we need to blacklist certain items in redux from being persisted (e.g. the app's connection status).
Debouncing the whole process of persistence is a simple and effective solution. Unfortunately, `redux-persist` dropped `debounce` between v4 and v5, replacing it with `throttle`. `throttle`, instead of delaying the expensive action until a period of X ms of inactivity, simply ensures the action is executed at least every X ms. Of course, this does not fix our performance issue.
The patch is very simple. It adds a `debounce` argument - a number of milliseconds - and debounces `redux-persist`'s `update()` method (provided by `createPersistoid`) by that many ms.
Before this, I also tried writing a custom storage adapter for `redux-persist` to debounce the calls to `localStorage.setItem()`. While this worked and was far less invasive, it doesn't actually address the issue. It turns out `setItem()` is a very fast part of the process.
We use `redux-deep-persist` to simplify the `redux-persist` configuration, which can get complicated when you need to blacklist or whitelist deeply nested state. There is also a patch here for that library because it uses the same types as `redux-persist`.
Unfortunately, the last release of `redux-persist` used a package `flat-stream` which was malicious and has been removed from npm. The latest commits to `redux-persist` (about 1 year ago) do not build; we cannot use the master branch. And between the last release and last commit, the changes have all been breaking.
Patching this last release (about 3 years old at this point) directly is far simpler than attempting to fix the upstream library's master branch or figuring out an alternative to the malicious and now non-existent dependency.
2022-11-12 09:41:48 +00:00
|
|
|
var whitelist = config.whitelist || null;
|
|
|
|
|
var transforms = config.transforms || [];
|
|
|
|
|
var throttle = config.throttle || 0;
|
|
|
|
|
+ var debounce = config.debounce || 0;
|
|
|
|
|
var storageKey = "".concat(config.keyPrefix !== undefined ? config.keyPrefix : KEY_PREFIX).concat(config.key);
|
|
|
|
|
var storage = config.storage;
|
|
|
|
|
var serialize;
|
2022-11-12 10:47:05 +00:00
|
|
|
@@ -28,30 +30,37 @@ export default function createPersistoid(config) {
|
Patches redux-persist and redux-deep-persist with debounced persists
Our app changes redux state very, very often. As our undo/redo history grows, the calls to persist state start to take in the 100ms range, due to a the deep cloning of the history. This causes very noticeable performance lag.
The deep cloning is required because we need to blacklist certain items in redux from being persisted (e.g. the app's connection status).
Debouncing the whole process of persistence is a simple and effective solution. Unfortunately, `redux-persist` dropped `debounce` between v4 and v5, replacing it with `throttle`. `throttle`, instead of delaying the expensive action until a period of X ms of inactivity, simply ensures the action is executed at least every X ms. Of course, this does not fix our performance issue.
The patch is very simple. It adds a `debounce` argument - a number of milliseconds - and debounces `redux-persist`'s `update()` method (provided by `createPersistoid`) by that many ms.
Before this, I also tried writing a custom storage adapter for `redux-persist` to debounce the calls to `localStorage.setItem()`. While this worked and was far less invasive, it doesn't actually address the issue. It turns out `setItem()` is a very fast part of the process.
We use `redux-deep-persist` to simplify the `redux-persist` configuration, which can get complicated when you need to blacklist or whitelist deeply nested state. There is also a patch here for that library because it uses the same types as `redux-persist`.
Unfortunately, the last release of `redux-persist` used a package `flat-stream` which was malicious and has been removed from npm. The latest commits to `redux-persist` (about 1 year ago) do not build; we cannot use the master branch. And between the last release and last commit, the changes have all been breaking.
Patching this last release (about 3 years old at this point) directly is far simpler than attempting to fix the upstream library's master branch or figuring out an alternative to the malicious and now non-existent dependency.
2022-11-12 09:41:48 +00:00
|
|
|
var timeIterator = null;
|
|
|
|
|
var writePromise = null;
|
|
|
|
|
|
|
|
|
|
- var update = function update(state) {
|
2022-11-12 10:47:05 +00:00
|
|
|
- // add any changed keys to the queue
|
|
|
|
|
- Object.keys(state).forEach(function (key) {
|
|
|
|
|
- if (!passWhitelistBlacklist(key)) return; // is keyspace ignored? noop
|
|
|
|
|
+ // Timer for debounced `update()`
|
|
|
|
|
+ let timer = 0;
|
|
|
|
|
|
|
|
|
|
- if (lastState[key] === state[key]) return; // value unchanged? noop
|
|
|
|
|
+ function update(state) {
|
|
|
|
|
+ // Debounce the update
|
|
|
|
|
+ clearTimeout(timer);
|
|
|
|
|
+ timer = setTimeout(() => {
|
|
|
|
|
+ // add any changed keys to the queue
|
|
|
|
|
+ Object.keys(state).forEach(function (key) {
|
|
|
|
|
+ if (!passWhitelistBlacklist(key)) return; // is keyspace ignored? noop
|
|
|
|
|
|
|
|
|
|
- if (keysToProcess.indexOf(key) !== -1) return; // is key already queued? noop
|
|
|
|
|
+ if (lastState[key] === state[key]) return; // value unchanged? noop
|
|
|
|
|
|
|
|
|
|
- keysToProcess.push(key); // add key to queue
|
|
|
|
|
- }); //if any key is missing in the new state which was present in the lastState,
|
|
|
|
|
- //add it for processing too
|
|
|
|
|
+ if (keysToProcess.indexOf(key) !== -1) return; // is key already queued? noop
|
|
|
|
|
|
|
|
|
|
- Object.keys(lastState).forEach(function (key) {
|
|
|
|
|
- if (state[key] === undefined && passWhitelistBlacklist(key) && keysToProcess.indexOf(key) === -1 && lastState[key] !== undefined) {
|
|
|
|
|
- keysToProcess.push(key);
|
|
|
|
|
- }
|
|
|
|
|
- }); // start the time iterator if not running (read: throttle)
|
|
|
|
|
+ keysToProcess.push(key); // add key to queue
|
|
|
|
|
+ }); //if any key is missing in the new state which was present in the lastState,
|
|
|
|
|
+ //add it for processing too
|
|
|
|
|
|
|
|
|
|
- if (timeIterator === null) {
|
|
|
|
|
- timeIterator = setInterval(processNextKey, throttle);
|
|
|
|
|
- }
|
|
|
|
|
+ Object.keys(lastState).forEach(function (key) {
|
|
|
|
|
+ if (state[key] === undefined && passWhitelistBlacklist(key) && keysToProcess.indexOf(key) === -1 && lastState[key] !== undefined) {
|
|
|
|
|
+ keysToProcess.push(key);
|
|
|
|
|
+ }
|
|
|
|
|
+ }); // start the time iterator if not running (read: throttle)
|
Patches redux-persist and redux-deep-persist with debounced persists
Our app changes redux state very, very often. As our undo/redo history grows, the calls to persist state start to take in the 100ms range, due to a the deep cloning of the history. This causes very noticeable performance lag.
The deep cloning is required because we need to blacklist certain items in redux from being persisted (e.g. the app's connection status).
Debouncing the whole process of persistence is a simple and effective solution. Unfortunately, `redux-persist` dropped `debounce` between v4 and v5, replacing it with `throttle`. `throttle`, instead of delaying the expensive action until a period of X ms of inactivity, simply ensures the action is executed at least every X ms. Of course, this does not fix our performance issue.
The patch is very simple. It adds a `debounce` argument - a number of milliseconds - and debounces `redux-persist`'s `update()` method (provided by `createPersistoid`) by that many ms.
Before this, I also tried writing a custom storage adapter for `redux-persist` to debounce the calls to `localStorage.setItem()`. While this worked and was far less invasive, it doesn't actually address the issue. It turns out `setItem()` is a very fast part of the process.
We use `redux-deep-persist` to simplify the `redux-persist` configuration, which can get complicated when you need to blacklist or whitelist deeply nested state. There is also a patch here for that library because it uses the same types as `redux-persist`.
Unfortunately, the last release of `redux-persist` used a package `flat-stream` which was malicious and has been removed from npm. The latest commits to `redux-persist` (about 1 year ago) do not build; we cannot use the master branch. And between the last release and last commit, the changes have all been breaking.
Patching this last release (about 3 years old at this point) directly is far simpler than attempting to fix the upstream library's master branch or figuring out an alternative to the malicious and now non-existent dependency.
2022-11-12 09:41:48 +00:00
|
|
|
+
|
2022-11-12 10:47:05 +00:00
|
|
|
+ if (timeIterator === null) {
|
|
|
|
|
+ timeIterator = setInterval(processNextKey, throttle);
|
|
|
|
|
+ }
|
Patches redux-persist and redux-deep-persist with debounced persists
Our app changes redux state very, very often. As our undo/redo history grows, the calls to persist state start to take in the 100ms range, due to a the deep cloning of the history. This causes very noticeable performance lag.
The deep cloning is required because we need to blacklist certain items in redux from being persisted (e.g. the app's connection status).
Debouncing the whole process of persistence is a simple and effective solution. Unfortunately, `redux-persist` dropped `debounce` between v4 and v5, replacing it with `throttle`. `throttle`, instead of delaying the expensive action until a period of X ms of inactivity, simply ensures the action is executed at least every X ms. Of course, this does not fix our performance issue.
The patch is very simple. It adds a `debounce` argument - a number of milliseconds - and debounces `redux-persist`'s `update()` method (provided by `createPersistoid`) by that many ms.
Before this, I also tried writing a custom storage adapter for `redux-persist` to debounce the calls to `localStorage.setItem()`. While this worked and was far less invasive, it doesn't actually address the issue. It turns out `setItem()` is a very fast part of the process.
We use `redux-deep-persist` to simplify the `redux-persist` configuration, which can get complicated when you need to blacklist or whitelist deeply nested state. There is also a patch here for that library because it uses the same types as `redux-persist`.
Unfortunately, the last release of `redux-persist` used a package `flat-stream` which was malicious and has been removed from npm. The latest commits to `redux-persist` (about 1 year ago) do not build; we cannot use the master branch. And between the last release and last commit, the changes have all been breaking.
Patching this last release (about 3 years old at this point) directly is far simpler than attempting to fix the upstream library's master branch or figuring out an alternative to the malicious and now non-existent dependency.
2022-11-12 09:41:48 +00:00
|
|
|
|
2022-11-12 10:47:05 +00:00
|
|
|
- lastState = state;
|
|
|
|
|
+ lastState = state;
|
|
|
|
|
+ }, debounce)
|
|
|
|
|
};
|
Patches redux-persist and redux-deep-persist with debounced persists
Our app changes redux state very, very often. As our undo/redo history grows, the calls to persist state start to take in the 100ms range, due to a the deep cloning of the history. This causes very noticeable performance lag.
The deep cloning is required because we need to blacklist certain items in redux from being persisted (e.g. the app's connection status).
Debouncing the whole process of persistence is a simple and effective solution. Unfortunately, `redux-persist` dropped `debounce` between v4 and v5, replacing it with `throttle`. `throttle`, instead of delaying the expensive action until a period of X ms of inactivity, simply ensures the action is executed at least every X ms. Of course, this does not fix our performance issue.
The patch is very simple. It adds a `debounce` argument - a number of milliseconds - and debounces `redux-persist`'s `update()` method (provided by `createPersistoid`) by that many ms.
Before this, I also tried writing a custom storage adapter for `redux-persist` to debounce the calls to `localStorage.setItem()`. While this worked and was far less invasive, it doesn't actually address the issue. It turns out `setItem()` is a very fast part of the process.
We use `redux-deep-persist` to simplify the `redux-persist` configuration, which can get complicated when you need to blacklist or whitelist deeply nested state. There is also a patch here for that library because it uses the same types as `redux-persist`.
Unfortunately, the last release of `redux-persist` used a package `flat-stream` which was malicious and has been removed from npm. The latest commits to `redux-persist` (about 1 year ago) do not build; we cannot use the master branch. And between the last release and last commit, the changes have all been breaking.
Patching this last release (about 3 years old at this point) directly is far simpler than attempting to fix the upstream library's master branch or figuring out an alternative to the malicious and now non-existent dependency.
2022-11-12 09:41:48 +00:00
|
|
|
|
|
|
|
|
function processNextKey() {
|
|
|
|
|
diff --git a/node_modules/redux-persist/es/types.js.flow b/node_modules/redux-persist/es/types.js.flow
|
|
|
|
|
index c50d3cd..39d8be2 100644
|
|
|
|
|
--- a/node_modules/redux-persist/es/types.js.flow
|
|
|
|
|
+++ b/node_modules/redux-persist/es/types.js.flow
|
|
|
|
|
@@ -19,6 +19,7 @@ export type PersistConfig = {
|
|
|
|
|
whitelist?: Array<string>,
|
|
|
|
|
transforms?: Array<Transform>,
|
|
|
|
|
throttle?: number,
|
|
|
|
|
+ debounce?: number,
|
|
|
|
|
migrate?: (PersistedState, number) => Promise<PersistedState>,
|
|
|
|
|
stateReconciler?: false | Function,
|
|
|
|
|
getStoredState?: PersistConfig => Promise<PersistedState>, // used for migrations
|
|
|
|
|
diff --git a/node_modules/redux-persist/lib/types.js.flow b/node_modules/redux-persist/lib/types.js.flow
|
|
|
|
|
index c50d3cd..39d8be2 100644
|
|
|
|
|
--- a/node_modules/redux-persist/lib/types.js.flow
|
|
|
|
|
+++ b/node_modules/redux-persist/lib/types.js.flow
|
|
|
|
|
@@ -19,6 +19,7 @@ export type PersistConfig = {
|
|
|
|
|
whitelist?: Array<string>,
|
|
|
|
|
transforms?: Array<Transform>,
|
|
|
|
|
throttle?: number,
|
|
|
|
|
+ debounce?: number,
|
|
|
|
|
migrate?: (PersistedState, number) => Promise<PersistedState>,
|
|
|
|
|
stateReconciler?: false | Function,
|
|
|
|
|
getStoredState?: PersistConfig => Promise<PersistedState>, // used for migrations
|
|
|
|
|
diff --git a/node_modules/redux-persist/src/types.js b/node_modules/redux-persist/src/types.js
|
|
|
|
|
index c50d3cd..39d8be2 100644
|
|
|
|
|
--- a/node_modules/redux-persist/src/types.js
|
|
|
|
|
+++ b/node_modules/redux-persist/src/types.js
|
|
|
|
|
@@ -19,6 +19,7 @@ export type PersistConfig = {
|
|
|
|
|
whitelist?: Array<string>,
|
|
|
|
|
transforms?: Array<Transform>,
|
|
|
|
|
throttle?: number,
|
|
|
|
|
+ debounce?: number,
|
|
|
|
|
migrate?: (PersistedState, number) => Promise<PersistedState>,
|
|
|
|
|
stateReconciler?: false | Function,
|
|
|
|
|
getStoredState?: PersistConfig => Promise<PersistedState>, // used for migrations
|
|
|
|
|
diff --git a/node_modules/redux-persist/types/types.d.ts b/node_modules/redux-persist/types/types.d.ts
|
|
|
|
|
index b3733bc..2a1696c 100644
|
|
|
|
|
--- a/node_modules/redux-persist/types/types.d.ts
|
|
|
|
|
+++ b/node_modules/redux-persist/types/types.d.ts
|
|
|
|
|
@@ -35,6 +35,7 @@ declare module "redux-persist/es/types" {
|
|
|
|
|
whitelist?: Array<string>;
|
|
|
|
|
transforms?: Array<Transform<HSS, ESS, S, RS>>;
|
|
|
|
|
throttle?: number;
|
|
|
|
|
+ debounce?: number;
|
|
|
|
|
migrate?: PersistMigrate;
|
|
|
|
|
stateReconciler?: false | StateReconciler<S>;
|
|
|
|
|
/**
|