YouTube Videos

A Simple Neural Network
KotlinConf 2018 - Mathematical Modeling
Creating a Sudoku Solver from Scratch
Traveling Salesman Problem
Text Categorization w/ Naive Bayes
Monty Hall Problem
Solving World's Hardest Sudoku

Wednesday, January 3, 2018

Kotlin and Linear Programming Part I - Binary Programming

kotlin_for_operational_planning_and_optimization In November 2017, I spoke at KotlinConf proposing Kotlin as a data science platform. When I was planning the code examples for the presentation, there were many topics to choose from. A common example would be a “classic” machine learning solution processing images, text, or speech with a neural network. I probably could have done something with TensorFlow which would undoubtedly have been cool.
But as fascinating as that area of data science is, I strongly felt the need to do something different. I wanted to present a topic that is highly relevant and for unknown reasons gets shadowed. I wanted to present something practical, accessible, and excitingly useful all at the same time, which would be especially appropriate for a Kotlin event. That is why I chose linear/integer programming and presented a driver scheduling example using Kotlin and ojAlgo.
Linear programming (also called linear optimization) is an applied field of mathematics often used in operations research and planning. It attempts to find an optimal solution to a planning problem when a set of business constraints exist. Some areas where linear programming is critical include:
  • Transportation networks (airlines, trains, ground, etc)
  • Staff scheduling (nurses, doctors, workers, teachers)
  • Resource planning (classroom scheduling, manufacturing, farming, supply chain management)
  • Product mixing (financial portfolios, ingredient mixing, etc)
  • Game “AI” (sudoku, chess, tic-tac-toe, etc)
More succinctly, when you hear a business problem that includes objectives that say “minimize”, “maximize”, or simply find a solution that’s “feasible”, linear programming should come right out of your tool kit. As a matter of fact, a case can be made that constraint solving is a form of machine learning. In many situations, it may even be a better approach than neural networks or other forms of AI.
Linear programming is a quiet underdog and workhorse, used heavily in operation-driven industries with little fanfare. Maybe I have my biases since I work for an airline, but it surprises me how little attention it gets. Even for consumer mobile apps, there is a wide spectrum of apps that have never been explored. For instance, how many mobile apps are there that will automatically schedule staff shifts? There are certainly big, expensive enterprise applications out there, but I have not seen a $5 Android app that does this.

Linear vs Integer vs Mixed Programming

“Linear” means continuous and straight, which more or less means a variable can be be solved as a decimal value along a straight line. If I am scheduling a worker for a given day, the optimal shift start can solve to 12, 12.5 hours, and even 12.5000012 hours. The number is allowed to float freely anywhere on a continuous number line, rather than using fixed whole numbers. Pure linear programming may be desirable for “mixing problems”, such as food ingredients or financial portfolio optimization.
For scheduling problems however, pure linear programming is not enough. Shifts often are modeled as discrete “slots” which can be represented with whole numbers. For instance, I may have a 12-hour day and I need to assign workers for that day. I could arbitrarily decide a “slot” is 1 hour, which would result in 12 slots. I could make a slot 4 hours, which would result in 3 slots or “shifts”. My model then describes the problem as “Slot1, Slot2, Slot3”… and so on. This is known as integer, or “discrete” optimization.
If you do not want to optimize with partial units, integer programming is handy for that situation as well. For instance, I may not want to manufacture 100.5002 units of a product. It must be a whole number like 100 or 101. Integer programming is also often used for binary/boolean logic by limiting a variable to 1 or 0, which is what I will focus on heavily in this blog post.
You can mix linear and integer programming, and this is known as “mixed integer programming”. This basically means some variables being solved are allowed to be continuous, while others may be integers. The driver problem I presented at KotlinConf was a mixed problem, as the driver shift start/ends were continuous, but to prevent overlap I used binary variables that could be 1 or 0.
But again in this blog post, I am going to focus heavily on binary optimization. I am hoping to do a series of blog posts on linear programming and optimization, but I think this will make a good first topic.

Getting Set Up

To keep concepts focused and simple, I am going to present an abstract problem. However this can be applied to real-life examples especially for scheduling applications. I am going to use Kotlin and ojAlgo, so be sure to configure both in your Maven/Gradle dependencies.
Before we dive in, also add these helper functions for ojAlgo to act as an improvised DSL. We will also instantiate an ExpressionsBasedModel and our helper functions will automatically name variables and functions before adding them to it.
 
import org.ojalgo.optimisation.ExpressionsBasedModel
import org.ojalgo.optimisation.Variable
import java.util.concurrent.atomic.AtomicInteger
 
// declare ojAlgo Model
val model = ExpressionsBasedModel()
 
// custom DSL for  expression inputs, eliminate naming and adding
val funcId = AtomicInteger(0)
val variableId = AtomicInteger(0)
fun variable() = Variable(variableId.incrementAndGet().toString().let { "Variable$it" }).apply(model::addVariable)
fun addExpression() = funcId.incrementAndGet().let { "Func$it"}.let { model.addExpression(it) }

An Abstract Example

Say we have five letters: A, B, C, D, and E. Let’s represent this as a Kotlin enumerable.
enum class Letter {
    A,
    B,
    C,
    D,
    E;
}
Now let’s create five numbers: ONE, TWO, THREE, FOUR, and FIVE. Please do not confuse this with the Java Number type. We are defining our own. The value property will hold its int counterpart.
enum class Number(val value: Int)  {
    ONE(1),
    TWO(2),
    THREE(3),
    FOUR(4),
    FIVE(5);
 
    override fun toString() = value.toString()
}
This pattern we are about to learn is useful and can be used to assign university classes, employee schedule shifts, etc. For example, Letter could represent an employee, and Number could be a shift. But let’s keep this abstract for now.
Say we want to assign each Letter to a Number, but it can only be assigned once. Now we could do this simply with a zip() operation, but we are going to make this problem subject to additonal constraints and rules later. Therefore, let’s do this mathematically and we will build on that foundation.
Envision a grid much like that board game Battleship where one axis is each Letter and the other is each Number. The intersection of each Letter or Number will be a 1 or 0 (true or false), indicating whether that Letter was assigned to that Number. Here is one possible solution done visually:
ONE TWO THREE FOUR FIVE
A 0 1 0 0 0
B 0 0 0 1 0
C 0 0 1 0 0
D 1 0 0 0 0
E 0 0 0 0 1
Bear with me. Let’s represent each “intersection” as a Slot class, which holds that given Letter and Number. We will also create an ojAlgo Variable that is binary, meaning it is constrained to be 1 or 0. We will also create a companion object to build and hold all the instances shown in this grid.
data class Slot(val letter: Letter, val number: Number) {
 
    val occupied = variable().binary()
 
    companion object {
        val all = Letter.values().asSequence().flatMap { letter ->
            Number.values().asSequence().map { number -> Slot(letter, number) }
        }.toList()
    }
    override fun toString() = "$letter$number: ${occupied?.value?.toInt()}"
}
How do you mathematically enforce a Letter only getting assigned to a Number once, and vice versa? Easy! To see if a letter has only been assigned once, just sum all of its “slot” values and enforce that it must not be greater than 1. Visually, all the values in a given column or a given row must sum to no more than 1. Mathematically, this would be expressed like this:

We could also use = instead of <=, but if we ever had more numbers than letters perhaps we would be okay with no assignment happening at all. We just do not want it assigned more than once.
Do the same for each Number. For instance, ONE's slots must sum to no more than 1.

To execute this simple logic, modify the Letter and Number classes to retrieve their slots. Then we will append an addConstraints() function to each one, where we will call addExpression(), set the upper() to 1, and then loop through each slot to “add” its variable. Since we have no need to multiply each variable with a coefficient, just set it to 1 as multiplying 1 will have no effect.
enum class Letter {
    A,
    B,
    C,
    D,
    E;
 
    val slots by lazy {
        Slot.all.filter { it.letter == this }.sortedBy { it.number.value }
    }
 
    fun addConstraints() {
 
        // constrain each letter to only be assigned once
        addExpression().upper(1).apply {
            slots.forEach {
                set(it.occupied, 1) // multiplier of 1
            }
        }
    }
}
 
enum class Number(val value: Int)  {
    ONE(1),
    TWO(2),
    THREE(3),
    FOUR(4),
    FIVE(5);
 
    val slots by lazy {
        Slot.all.filter { it.number == this }.sortedBy { it.letter }
    }
 
    fun addConstraints() {
 
        // constrain each NUMBER to only be assigned one slot
        addExpression().upper(1).apply {
            slots.forEach {
                set(it.occupied, 1) //multiplier of 1
            }
        }
    }
 
    override fun toString() = value.toString()
}
If that set() function is confusing you, we are basically just multiplying each binary by 1 which has no effect.

Now let’s run this. We will loop through each Letter and Number to call its addConstraints() function. We are not trying to minimize() or maximize() anything (such as profit or cost) in this problem. We just want something that is operable and works with all constraints we defined. Then we will print a pretty grid of all our results.
fun main(args: Array<String>) {
 
    Letter.values().forEach { it.addConstraints() }
    Number.values().forEach { it.addConstraints() }
 
    model.minimise().run(::println)
 
    Letter.values().joinToString(prefix = "   ", separator = "   ").run(::println)
 
    Number.values().forEach { n ->
        Letter.values().asSequence().map { l -> l.slots.first { it.number == n }.occupied.value.toInt() }
                .joinToString(prefix = "$n  ", separator = "   ").run { println(this) }
    }
}
Here is the output I got:
OPTIMAL 0.0 @ [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0]
   A   B   C   D   E
1  0   0   0   1   0
2  0   0   1   0   0
3  0   1   0   0   0
4  1   0   0   0   0
5  0   0   0   0   1
As you can see, we got an OPTIMAL result meaning it found a solution (as opposed to an INFEASIBLE if no solution is possible with our constraints). As you can see in our improvised console grid, each letter was cleanly assigned (via a 1) to a Number.
Now let’s jump this up a notch.

Restricting Items to Specific Regions

Say two additional rules need to be added to our model:
  • C must be assigned to a Number greater than or equal to THREE
  • D must be assigned to a Number less than or equal to TWO
These fixed constraints are usually not hard to implement. It is when we start making variables relative to other variables that things start to get gnarly (covered later in this article). But for these two rules, all we have to do is tease out the slots in those desired ranges, and say they must sum to 1.

We can add these two constraints to our model, and at least one of those slots for C and D respectively must be 1 for those equations to yield true. Let’s just do this in our main() function. We are going to use level() instead of upper() this to express a strict = rather than a <=.
fun main(args: Array<String>) {
 
    Letter.values().forEach { it.addConstraints() }
    Number.values().forEach { it.addConstraints() }
 
    // C must be greater than or equal to THREE
    addExpression().level(1).apply {
        Letter.C.slots.asSequence().filter { it.number.value >= 3 }.forEach {
            set(it.occupied, 1)
        }
    }
 
    // D must be less than or equal to TWO
    addExpression().level(1).apply {
        Letter.D.slots.asSequence().filter { it.number.value <= 2 }.forEach {
            set(it.occupied, 1)
        }
    }
 
 
    model.minimise().run(::println)
 
    Letter.values().joinToString(prefix = "   ", separator = "   ").run(::println)
 
    Number.values().forEach { n ->
        Letter.values().asSequence().map { l -> l.slots.first { it.number == n }.occupied.value.toInt() }
                .joinToString(prefix = "$n  ", separator = "   ").run { println(this) }
    }
}
When we run this, sure enough we get an output that will conform to those two new rules:
OPTIMAL 0.0 @ [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0]
   A   B   C   D   E
1  1   0   0   0   0
2  0   0   0   1   0
3  0   0   1   0   0
4  0   1   0   0   0
5  0   0   0   0   1
Alright, let’s escalate this into something more realistic.

Using Multiple Consecutive Blocks

When you start dealing with relative constraints, this is where things become less intuitive. Linear and integer programming limits you to only express functions in a simple addition/subtraction structure such as Ax + By + Cz <= 40 (A, B, and C are constants multiplied to variables x, y, and z). You cannot even do simple operations like multiply variables to other variables or use if expressions in the model functions. Therefore, you really have to challenge yourself to think creatively with pencil and paper, and not be afraid to experiment with mathematical equations and proofs to solve your problem before you even write code.
For instance, let’s say we want to add a concept to our model where a Letter can consume multiple contiguous slots. This is realistic as classrooms, staff, and other scheduling problems may break shifts up into say, 15 minute blocks, and a classroom can be scheduled for 90 minutes (or 6 blocks). In this case, let’s add the following rules:
  • Letter B requires 2 contiguous slots
  • Letter C requires 3 contiguous slots
The first thing we will need to do is add more Number items, as we do not have enough for these new requirements. Let’s add all the way up to NINE, even if it is actually more than we need (just because).
enum class Number(val value: Int)  {
    ONE(1),
    TWO(2),
    THREE(3),
    FOUR(4),
    FIVE(5),
    SIX(6),
    SEVEN(7),
    EIGHT(8),
    NINE(9);
 
    val slots by lazy {
        Slot.all.filter { it.number == this }.sortedBy { it.letter }
    }
 
    fun addConstraints() {
 
        // constrain each NUMBER to only be assigned one slot
        addExpression().upper(1).apply {
            slots.forEach {
                set(it.occupied, 1)
            }
        }
    }
 
    override fun toString() = value.toString()
}
So how in the world do we express this logic of multiple contiguous blocks as a linear function? It is not intuitive so the best place to start is to try and make it intuitive. Let’s plot with pencil and paper some valid scenarios for what we want. Here is an example of a desired scenario:
ONE TWO THREE FOUR FIVE SIX SEVEN EIGHT NINE
A 0 1 0 0 0 0 0 0 0
B 0 0 0 1 1 0 0 0 0
C 0 0 0 0 0 1 1 1 0
D 1 0 0 0 0 0 0 0 0
E 0 0 0 0 0 0 0 0 1
Let’s focus just on the row for C. Since C requires three contiguous slots, let’s break down a given case into rolling groups of three like this:
ONE TWO THREE FOUR FIVE SIX SEVEN EIGHT NINE
ALL 0 0 0 0 0 1 1 1 0
GROUP 1 0 0 0
GROUP 2 0 0 0
GROUP 3 0 0 0
GROUP 4 0 0 1
GROUP 5 0 1 1
GROUP 6 1 1 1
GROUP 7 1 1 0
Now we have rolling groups of three slots. GROUP 6 has the desired set of three contiguous 1’s. Let’s label each of these groups with a RESULT column, where 1 indicates a success and 0 indicates a fail like this:
RESULT ONE TWO THREE FOUR FIVE SIX SEVEN EIGHT NINE
ALL 0 0 0 0 0 1 1 1 0
GROUP 1 0 0 0
GROUP 2 0 0 0 0
GROUP 3 0 0 0 0
GROUP 4 0 0 0 1
GROUP 5 0 0 1 1
GROUP 6 1 1 1 1
GROUP 7 0 1 1 0
So what we really want is a series of constraints expressed as these equations:

Well that escalated quickly. Now before you throw in the towel, really study what is happening above and give it a chance to sink in. Pull out a pencil and paper if you need to and proof a few cases. Again, we broke C’s slots into contiguous, rolling groups of three. One group needs to result in a 1 while the rest must be a 0. The series of functions above express this. We create a new binary variable for each group that can be 1 or 0. We then add all those binaries together which should sum to 1, meaning only one of them must be 1 while the rest are 0.
The M is actually going to be 3 in this case, because we need three slots. We multiply it to the group’s binary and subtract it as a subtle way of executing a constraint. That expression must be greater than or equal to 0, and that M may or may not be voided if the group’s binary is 1 or 0. If we have 1’s scattered across multiple groups, the entire system of equations will invalidate that case. If three contiguous 1’s exist in a group, that will satisfy the system of equations.
Here is how we express this in our Kotlin model:
enum class Letter(val slotsNeeded: Int = 1) {
    A,
    B(slotsNeeded = 2),
    C,
    D(slotsNeeded = 3),
    E;
 
    val slots by lazy {
        Slot.all.filter { it.letter == this }.sortedBy { it.number.value }
    }
 
    fun addConstraints() {
 
        // constrain each LETTER to number of slots needed
        addExpression().level(slotsNeeded).apply {
            slots.forEach {
                set(it.occupied, 1)
            }
        }
 
        //only inflict model with contiguous logic if needed
        if (slotsNeeded > 1) {
 
            val allGroupSlots = addExpression().level(1)
 
            slots.rollingBatches(slotsNeeded).forEach { group ->
 
                val slotForGroup = variable().binary()
 
                allGroupSlots.set(slotForGroup, 1)
 
                addExpression().lower(0).apply {
                    group.forEach {
                        set(it.occupied,1)
                    }
                    set(slotForGroup, -1 * slotsNeeded)
                }
            }
        }
    }
}
When you run this, sure enough you get contiguous sets of slots occupied for B and C.
OPTIMAL 0.0 @ [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0]
   A   B   C   D   E
1  0   0   0   0   0
2  0   0   0   1   0
3  0   0   0   1   0
4  0   0   0   1   0
5  0   0   1   0   0
6  0   1   0   0   0
7  0   1   0   0   0
8  1   0   0   0   0
9  0   0   0   0   1
Here is all the code in its entirety. You can also clone the code from GitHub.
import org.ojalgo.optimisation.ExpressionsBasedModel
import org.ojalgo.optimisation.Variable
import java.util.concurrent.atomic.AtomicInteger
 
// declare ojAlgo Model
val model = ExpressionsBasedModel()
 
// custom DSL for Example3.getModel expression inputs, eliminate naming and adding
val funcId = AtomicInteger(0)
val variableId = AtomicInteger(0)
fun variable() = Variable(variableId.incrementAndGet().toString().let { "Variable$it" }).apply(model::addVariable)
fun addExpression() = funcId.incrementAndGet().let { "Func$it"}.let { model.addExpression(it) }
 
 
fun main(args: Array<String>) {
 
    Letter.values().forEach { it.addConstraints() }
    Number.values().forEach { it.addConstraints() }
 
    // C must be greater than or equal to THREE
    addExpression().level(1).apply {
        Letter.C.slots.asSequence().filter { it.number.value >= 3 }.forEach {
            set(it.occupied, 1)
        }
    }
 
    // D must be less than or equal to TWO
    addExpression().level(1).apply {
        Letter.D.slots.asSequence().filter { it.number.value <= 2 }.forEach {
            set(it.occupied, 1)
        }
    }
 
 
    model.minimise().run(::println)
 
    Letter.values().joinToString(prefix = "   ", separator = "   ").run(::println)
 
    Number.values().forEach { n ->
        Letter.values().asSequence().map { l -> l.slots.first { it.number == n }.occupied.value.toInt() }
                .joinToString(prefix = "$n  ", separator = "   ").run { println(this) }
    }
}
 
enum class Letter(val slotsNeeded: Int = 1) {
    A,
    B(slotsNeeded = 2),
    C,
    D(slotsNeeded = 3),
    E;
 
    val slots by lazy {
        Slot.all.filter { it.letter == this }.sortedBy { it.number.value }
    }
 
    fun addConstraints() {
 
        // constrain each LETTER to number of slots needed
        addExpression().level(slotsNeeded).apply {
            slots.forEach {
                set(it.occupied, 1)
            }
        }
 
        // constrain slots to be consecutive if more than one are needed
        // this is tricky because slots need to be "batched" in rolling groups of needed slot size
        // e.g. for a slot size three, we need (x1,x2,x3), (x2,x3,x4), (x3,x4,x5) and so on. A binary is attached to each group
        // and another binary needs to be shared across all the batches
        // x1 + x2 + .. xi - Sb >= 0
        // Where xi is slot binaries, S is number of slots needed, and b is shared binary across all groups
        if (slotsNeeded > 1) {
 
            val allGroupSlots = addExpression().level(1)
 
            slots.rollingBatches(slotsNeeded).forEach { group ->
 
                val slotForGroup = variable().binary()
 
                allGroupSlots.set(slotForGroup, 1)
 
                addExpression().lower(0).apply {
                    group.forEach {
                        set(it.occupied,1)
                    }
                    set(slotForGroup, -1 * slotsNeeded)
                }
            }
        }
    }
}
 
enum class Number(val value: Int)  {
    ONE(1),
    TWO(2),
    THREE(3),
    FOUR(4),
    FIVE(5),
    SIX(6),
    SEVEN(7),
    EIGHT(8),
    NINE(9);
 
    val slots by lazy {
        Slot.all.filter { it.number == this }.sortedBy { it.letter }
    }
 
    fun addConstraints() {
 
        // constrain each NUMBER to only be assigned one slot
        addExpression().upper(1).apply {
            slots.forEach {
                set(it.occupied, 1)
            }
        }
    }
 
    override fun toString() = value.toString()
}
 
data class Slot(val letter: Letter, val number: Number) {
 
    val occupied = variable().binary()
 
    companion object {
        val all = Letter.values().asSequence().flatMap { letter ->
            Number.values().asSequence().map { number -> Slot(letter, number) }
        }.toList()
    }
    override fun toString() = "$letter$number: ${occupied?.value?.toInt()}"
}
 
fun <T> List<T>.rollingBatches(batchSize: Int) = (0..size).asSequence().map { i ->
    subList(i, (i + batchSize).let { if (it > size) size else it })
}.filter { it.size == batchSize }

Conclusions

Linear programming is fairly easy to model in most cases, but as you can see here integer/binary programming can be hard. I am still learning binary patterns to this day, and find it helpful to spend a lot of time with pencil and paper experimenting with equations. I find it is a hit-or-miss process with lots of proof work, and I hope to get proficient in tackling even larger real-world problems.
Thankfully Kotlin made it easy, organized, and refactorable to execute, and allowed us to spend more time thinking about the math. As you probably have noticed, the Kotlin stdlib came incredibly handy in preparing data for the model.
If you are dealing with thousands of variables, you may encounter performance issues even with ojAlgo. Thankfully most solvers, including ojAlgo, support plugging in commercial optimizer implementations like CPLEX. When your solver is acting exhausted or slow, and you are sure there is no modeling mistake, it may be worthwhile getting a commercial-grade solver.
In the coming few months, I will do a few more posts on using Kotlin for linear/integer programming and optimization. Until then, be sure to check out this book as well as this one. Also be sure to use Math Stack Exchange for specific modeling questions, and StackOverflow for ojAlgo questions.

71 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. This blog is truly useful to convey overhauled instructive undertakings over web which is truly examination. I discovered one fruitful case of this truth through this blog. I will utilize such data now.
    Cisco SG350X

    ReplyDelete
  3. Hello, I also would like to comment over all the points mentioned in this blog. I agree with essence of few point but somewhere I found myself on other place. I hope, there might little opinion of others as well.
    Lenovo ThinkSystem ST550

    ReplyDelete
  4. Somewhere the content of the blog surrounded by little arguments. Yes it is healthy for readers. They can include this kind of language in their writing skill as well as while group discussion in college.
    กลโกงไพ่

    ReplyDelete
  5. Nature always helps a writer to learn from its elements. Nature always try to teach new lessons and a writer transform its experience into words. He can transform even silence into words. This blog taught me a lesson.
    TeraStation 3410DN Series

    ReplyDelete
  6. This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good work recover lost bitcoins

    ReplyDelete
  7. Thanks for this post. I really appreciate your efforts. If anyone is looking for best Social Media Optimisation service in Australia, visit my profile.

    ReplyDelete

  8. You there, this is really good post here. Thanks for taking the time to post such valuable information. Quality content is what always gets the visitors coming.

    Apple Repair Dubai

    ReplyDelete
  9. Great Article it its really informative and innovative keep us posted with new updates. its was really valuable. Thank you a lot! and if you need App development company then contact us!

    ReplyDelete
  10. Hello!
    your post really awesome and informative article, thanks for sharing this information with us, keep posting, By the way, If anyone searching for Best MS Surface Pro Repair. Visit my profile

    ReplyDelete
  11. Excellent post – thanks for sharing the list of such amazing blogs, including yours.

    ReplyDelete
  12. Great post,visit my site for the problem solving services like; Home sharing error 5507

    ReplyDelete
  13. If you do not want to optimize with partial units, integer programming is handy for that situation as well.

    ReplyDelete
  14. Hello!
    Your post really fantastic and informative blog. Thanks for sharing this information with learner. If you want to change Keyboard Replacement kits available for best QWERTY US-English Version layout keyboards.

    ReplyDelete
  15. I am confident my friends get benefited from this sit Data Recovery Dubai

    ReplyDelete
  16. Hello dear,
    This is really good post here. Thanks for taking the time to post such valuable information for users. By the way, If anyone looking for Recruitment Services in Iraq. Visit Here

    ReplyDelete
  17. Hey, your post is awesome and if you want to grab data recovery related opportunity call us at

    ReplyDelete
  18. Thanks for sharing this post on Kotlin and Linear Programming. I was searching for this post to learn about Linear programming!

    ReplyDelete
  19. Hello Dear!
    Legal consultants will design such benefits package so that employees stick to your organization for a longer time. Lawyers in Dubai

    ReplyDelete
  20. Thanks for providing recent updates regarding the concern, I look forward to read more.
    代做cs

    ReplyDelete
  21. I’m really happy to find out this amazing blog, visit OGEN Infosystem for creative website designing and SEO Service in Delhi.
    SEO Service in Delhi

    ReplyDelete
  22. I am very glad to find this post, because it is very useful and helpful for us. This Post contain lots of information in it. Thanks for sharing the best article post. Website design Dubai , Website development Dubai, SEO service Dubai

    ReplyDelete
  23. We Fix All maps device issues, Magellan GPS update, Garmin map update, Tomtom map update or any other maps device.

    Garmin map updates
    Magellan GPS update ,

    ReplyDelete
  24. Hello!
    Thanks for share information about How to solve programming issues. This is an amazing techniques. Great work.

    ReplyDelete
  25. We help you to prepare your academic and college assignments. Get online assignment help by expert writers to secure good grades.
    do my assignment

    ReplyDelete
  26. I truly like to read your post. Thank you so much for taking the time to share such nice information.
    best mobile app development services in pakistan 2020

    ReplyDelete
  27. I as of late went over your site and have been perusing along. I thought I would leave my first remark. I don't recognize what to say with the exception of that I have delighted in perusing. Decent blog, I will continue going by this online journal all the time.
    Itel Mobile Price in Bangladesh

    ReplyDelete
  28. Hello guys, looking to enjoy a Delhi escort with fantastic body shape? I am Lovely Rani, a passionate Delhi babe, ready to fulfill all your dreams and aspirations.

    Delhi call girl

    ReplyDelete
  29. Pretty nice post. I just stumbled upon your weblog and wanted to say that I have really enjoyed browsing your blog posts. After all I’ll be subscribing to your feed and I hope you write again soon!
    cs作业 代写

    ReplyDelete
  30. Wow!!! It was really an Informational Article which provide me with much Insightful Information. I would like to first thank the author for such an Insightful Content. If someone here is looking for a Website Designing Company in India, then look no further than Jeewangarg.com as we are the best website designing company in Delhi working in the arena for the last 8 years.
    Best website designing company in India
    website designing company in NCR

    ReplyDelete
  31. Wow!!! A great insightful & informative content on ACP Board. Virgo Group is the largest providers of best in class leading facade materials and more & having years of experience in manufacturing Premium ACP Sheet. We aim to deliver excellence and ensure maximum worth for our clients’ money and so we provide you with the best ACP Sheet Price.

    ReplyDelete
  32. Thanks for the great article and post. Need a servo voltage stabilizer manufacturers in india? Don’t go beyond servo star “A popular 3 phase 10kva servo voltage stabilizer price which offers an extensive range of exceptional servo stabilizer. Feel Free to contact.

    ReplyDelete
  33. Thanks for the great article and post. Need a Industrial UPS? Don’t go beyond nexusups. “A popular Industrial UPS which offers an extensive range of exceptional Industrial UPS, and Online UPS Free to contact.

    ReplyDelete
  34. Thanks For Sharing this post. this post is very useful for me. also here Are you Looking for Indian Designer Anarkali dress or Anarkali Salwar Kameez anarkali?

    We have Largest & latest Collection of Indian Designer Anarkali Suits which is avilable now at Best Discounted Prices.

    ReplyDelete
  35. Thank You for providing us with such an insightful information through this blog. However, if you are in the search of finding Cab or Taxi Services in Melbourne? Your Search Ends here…
    silver service cabs is your answer for availing best silver taxi service in Melbourne. Reach us for the best Silver Service Cab Melbourne.

    ReplyDelete
  36. A list of the best ACP sheets and acp sheet manufacturers in India who produces compact, strong and durable sheets those are best for your home décor, office or Industry etc. Aluminium Composite Panel

    ReplyDelete
  37. I would first like to thank the author for coming up with the insightful information every time. People if you are also looking for the best shimla tour packages | Best Rajasthan Tour Packages then reach out to Indian Travel Store as we are the best Travel Partners of yours.

    ReplyDelete
  38. Well I definitely enjoyed reading it. This tip procured by you is very constructive for correct planning.
    video production
    film production

    ReplyDelete
  39. Really this is awsome post, which is so informative. We are the largest shot blasting machine manufacturers & Suppliers in India. Get the all types of sand blasting machine and their equipments @ best prices & unmatched services. Call us @ 9811083740.

    ReplyDelete
  40. Thanks for posting such a great post. mostly person have many doubt that the product which is puchaging through online is really right for us! is this better product? That's why right select is standing infront of you to provide right products. Right Select help to select right products.

    Out of them: Top 10 injection molding machine manufacturers in india

    ReplyDelete
  41. Imperial Garden is the best Banquet Hall in Faridabad. This banquet hall is located at near surajkund road Faridabad. It is a prestigious and well maintained hall for any function. Make your functions or your special moments more delightful with our best banquet hall and get yourself lost into the irresistible beauty of Imperial Garden.

    ReplyDelete
  42. We are providing the best Skin Specialist in Noida. Dermanext Skin & Hair Clinic is a boutique clinic specialising in dermatology, trichology and aesthetics. For more details call: 8860523726

    ReplyDelete
  43. We are providing the best Laptop Repair in Delhi. Also, our services are affordable and all the repairing work is performed by expert technicians. Get your computer repaired today. For more details call: 9868243838

    ReplyDelete
  44. We are providing the one of the best Web Development in Noida. For more details call: 9015664619

    ReplyDelete
  45. We are providing the Best Body Piercings In Delhi. For more details call: 8745801112

    ReplyDelete
  46. We are providing the best Computer Accessories in Ghaziabad. Our accessories are checked, tested and come with warranty. Our vendors manufacture these accessories using quality. For more details call: 9205059199

    ReplyDelete
  47. We are providing the best services Trademark registration in Delhi. Our Company Deals with all kind of Services/Certification related with Trademark. For more details call: 9899829509

    ReplyDelete
  48. We are providing the best Architects in Ghaziabad. We are artists, we are designers, and we are here to make your space better than you could have ever imagined. For more details call us: 7011210410

    ReplyDelete
  49. We are providing the best Modular Kitchen Dealer in Noida. For more details call us: 9868333789

    ReplyDelete
  50. We are providing the best Skin Clinic in Noida. Dermanext Skin & Hair Clinic is a boutique clinic specializing in dermatology, trichology and aesthetics. For more details call: 8860523726

    ReplyDelete
  51. Such an amazing article thanks for sharing this

    You’ll love to see my site http://sandblastingmachine.in/blog-sand-blasting-machine-manufacturer/ as well.

    Keep posting.

    ReplyDelete
  52. We are providing the Best tattoo artist in Lajpat Nagar. For more details call: 8745801112

    ReplyDelete

  53. PLEASE READ!!!! PLEASE READ!!!! PLEASE READ!!!! PLEASE READ!!!!

    Hey Guys!!!Am so happy I got mine from Mike Fisher. My blank ATM card can withdraw € 2,000 daily. I got it from Her last week and now I have €14,000 for free. The blank ATM withdraws money from any ATM machines and there is no name on it, it is not traceable and now i have money for business and enough money for me and my family to live on . I am really happy i met Mike Fisher because i met two people before her and they took my money not knowing that they were scams. But am happy now. Mike Fisher sent the card through DHL and i got it in two days. Get your own card from her now she is not like other scammer pretending to have the ATM card, She is giving it out for to help people even if it is illegal but it helps a lot and no one ever gets caught. i'm grateful to Mike Fisher because she changed my story all of a sudden . The card works in all countries except, Mali and Nigeria. Mike Fisher email address is : int.hackers002@gmail.com

    ReplyDelete
  54. OMAC Provides Best industrial automation Training in Noida. Omac is one of the most authoritative industrial automation training institutes in Noida offering hands-on practical education and full job support with basic as well as excellent level industrial automation training courses.Call@9015516908.https://www.omac.in

    ReplyDelete
  55. Thanks for sharing useful information. I learned something new from your bog. Its very interesting and informative. keep updating. If you are looking for any Big Data related information, please visit our website Big Data training institute in Bangalore.

    ReplyDelete
  56. Customer Relationship Software can be industry-specific but in most cases, it’s feature specific. It depends on which problem you are facing and which CRM is capable to help you.Customer Relationship Management Software UAE

    ReplyDelete
  57. I most likely appreciating each and every bit of it. It is an absurd site and decent impart. I need to much appreciative. Essay writers online

    ReplyDelete
  58. Personally, I have different passions that drives me into living a healthy life. First of all, the desire to be physically fit has led me into the lane of achieving this.freelance writer for hire

    ReplyDelete
  59. Wow!!! It was a great article I came to know a lot about the travel by going through this article. If you’re also looking for a Top 10 Travel Companies in India, visit right select to gain the best possible information about the Best Travel Agencies in India
    .

    ReplyDelete
  60. Hi I am Misha Khan, I am a student. I am interested technology related information. I am study technology related subject. I am really love your information Its very good. I am Launched technology related site you want to visit my site

    ReplyDelete
  61. En este momento de la digitalización, la técnica habitual para hacer pagos a través del dinero ha envejecido. Sustituya sus viejas técnicas por otras más actuales utilizando las administraciones en línea para pagar directamente a través de sus saldos financieros. Lo valioso de la administración de cuotas versátil que utiliza un pequeño mensaje, que se ha desarrollado obviamente patria desde que surgieron los problemas de ingresos, es que no necesita obtener un teléfono celular, es suficiente tener lo que ellos llaman un "responsable". Los clientes deben seleccionar en la fase de estos bancos, de los cuales indicarán la cantidad y la cantidad de intercambios habilitados siguiendo los parámetros, para utilizar la instalación por SMS. Las personas que utilizan saldos en el Banco de Venezuela banco de venezuela En Linea obtienen y harán compromisos interbancarios. Como un enfoque para obtener ganancias por la función, es urgente estar asociado con el BDV En Linea, en el cual debe iniciar la entrega esencial. Para las personas que tienen un registro gratuito en el banco del tesoro En Linea, puede crear la instalación de SMS de hasta 1,000,000 de bolívares de manera consistente, para eso debe consolidar la función en la instalación telefónica de Tesoro en Tesoro en la web y enviar, retirar la separación bnc , un correo electrónico al 2383 con toda la palabra Pagar, y los principales ejemplos de las cuentas bancarias de objetivos, el número de teléfono del destinatario, el tipo de cliente (V, Correo electrónico o J), el número de tarjeta de personalidad del beneficiario, el número fue con una coma y además dos decimales y código por la tarjeta de clasificación que se elegirá. Banco Bicentenario Debe inscribirse a través de su aplicación banco bicentenario En Linea Payment para explotar el servicio. En ese momento, pagará enviando un SMS al número 7077 juntos utilizando todo el términoEl código QR del video modificado se puede utilizar para descargar videos en su teléfono inteligente. Este convertidor mp3 en línea gratuito permite cambiar sus grabaciones de YouTube a MP3, MP4, MP4 HD, AVI y AVI HD. La interfaz es básica donde simplemente necesita pegar la URL del video y elegir el diseño de rendimiento. El registro modificado se hará accesible en la interfaz para descargar. También está disponible un módulo de programa gratuito para descargar videos. Con este programa en línea, sus grabaciones de YouTube se De hecho, mis abuelos tenían correo electrónico ... durante los años 90.Inevitablemente, el custodio estaba harto de observar involuntariamente que mi correo electrónico llegara al registro de la escuela. Por fin me arrinconó y dijo: "¿Has intentado hotmail ?" Entonces lo miré. Además, en ese primer año, fui un individuo sancionador por el trastorno de correo electrónico gratuito basado en la web. whatsapp web needs no presentation. It is all around the most well known texting application and has the most number of clients., GIFs, recordings, reports, this element bodes well. It tends to be exceptionally dull to return and quest for recently shared media, consequently another propelled search include. whatsapp web beta

    ReplyDelete
  62. We are providing the best Computer and Mobile Accessories in Ghaziabad. Our accessories are checked, tested and come with warranty. Our vendors manufacture these accessories using quality.

    ReplyDelete
  63. Baby Driver JacketMay 17, 2020 at 11:29 PM

    This was an extremely wonderful post. Thanks for providing this info. Baby Driver Jacket

    ReplyDelete
  64. This is the best post I have ever seen. Very clear and simple. Mid-portion Is quite interesting though. Keep doing this. I will visit your site again.
    ghost in the shell jacket

    ReplyDelete
  65. i am browsing this website dailly , and get nice facts from here all the time .

    ReplyDelete
  66. ABC Assignment Help services is a well-established and well-known name in providing best database assignment help service. Our main goal is to help all the students in completing their academic assignment tasks with quality on given time.

    ReplyDelete